Search

Most states responded to the onset of the pandemic by temporarily granting teachers Emergency licenses. These licenses allowed teachers to work in classrooms without passing the typical licensure exams. Since then, several states have extended their use of Emergency licenses, raising questions about how these policies impact the composition of the teacher workforce and student outcomes. In this paper, we examine the result of these policies using data on multiple cohorts of Emergency licensed teachers (ELTs) who taught in Massachusetts between 2021 and 2023. We find that ELTs were slightly more likely to remain in the same school and in the teaching workforce than teachers from other entry routes. However, ELTs' students scored significantly lower on standardized tests in math and science than other students in the same school and same year. Our findings are at odds with earlier, more positive assessments of Emergency licensure in Massachusetts. Our updated results appear to be driven by more recent cohorts of ELTs, rather than the teachers who received Emergency licenses at the start of the pandemic. Overall, this study suggests policymakers should be cautious when drawing sweeping conclusions about the impacts of teacher licensure based solely on the earliest cohort of teachers who obtained pandemic-era Emergency licenses.

Departmentalized instruction, in which teachers specialize in one or more core subjects and instruct multiple groups of students in a day, has become increasingly prominent in elementary schools. Using 8 years of data from Massachusetts and a difference-in-differences design, we estimate the effects of departmentalization on student achievement. We find that departmentalization has positive effects in English language arts (ELA) and science and mixed evidence of positive effects in math. These positive effects are not driven by teacher productivity improvements: Consistent with prior findings on teacher specialization, teachers are less effective when specializing in math and no more effective in ELA than when teaching self-contained classrooms. Rather, consistent with the theoretical underpinnings for specialization, departmentalized schools tend to assign teachers to their stronger subjects.

Graduates of special education teacher education programs can teach in a range of special education settings, raising the potential that their training can occur in very different settings than where they find their first jobs. We follow 263 completers of Moderate Disabilities programs in Massachusetts from their field placements to their early-career teaching positions and study the characteristics of their field placements and the degree to which these are aligned with their early-career teaching positions. We also assess the degree to which alignment is associated with early-career teacher turnover. We found that many of these teachers student-taught in an inclusive setting but were hired into a self-contained special education setting and vice versa, and teachers who experienced this misalignment were more likely to leave the workforce early in their careers. Teachers who student taught with a supervising practitioner without a special education license were also more likely to leave early. Findings suggest that teachers training to educate students with learning disabilities should student teach in a setting that is aligned with where they are likely to be hired, and with a supervising practitioner who is trained in special education.

Given math's importance, states are adopting a range of K12 policies to improve math outcomes. As these and other policies unfold, policymakers need to monitor intended outcomes and ultimately gauge impact. They need answers to basic questions: Are more students taking and passing algebra and advanced math? Are performance gaps changing between student groups or schools? What are the characteristics of our math teachers? Are they changing? How are math teachers with different characteristics distributed across schools and students? Policymakers also need answers to more complex questions: Did a given policy have an effect? If so, why, under what conditions, and for whom? Were there any unintended consequences? With these kinds of questions in mind, this CALDER brief singles out the value of longitudinal, individual-level state administrative data by reviewing questions that can--and can't--be answered by the more limited data made publicly available by states. To illustrate, we focus on three states that are attempting to improve math outcomes and that enroll large numbers of students from historically marginalized student groups: California, Florida, and Texas.

We investigate two research questions using a recent statewide transition from paper to computer-based testing: first, the extent to which test mode effects found in prior studies can be eliminated in large-scale administration; and second, the degree to which online and paper assessments offer different information about underlying student ability. In contrast to the first test transition in Massachusetts, we find very small mode effects for a more recent transition, which may be attributable to an additional step matching on observable characteristics in the equating process. Second, we investigate the predictive evidence of validity for paper and online tests for predictions of future test scores and grades. We generally find minimal differences for the extent to which scores on paper tests can differentially predict future online versus paper test scores. Finally, online and paper test scores are similarly predictive of future grade point average. We conclude that the online test penalty can vary substantially by test and that extreme care should be taken when administering online tests to some students and paper tests to others.

Research in education, economics, and psychology suggests that teachers contribute in a variety of ways to the learning environment. They form relationships with students and help inculcate effective learning habits (Battistich et al., 1997; Davis, 2003; Thomas & Oldfather, 1997). Teaching practices mediate students' feelings of belonging and engagement (Liu & Loeb, 2021; Rowley et al., 2019). They manage classroom behavior and establish expectations for students' work (Ferguson & Danielson, 2014; Kane et al., 2011). These teaching skills contribute to student academic outcomes in the short run and have lasting effects on the educational trajectories of students (Blazar & Kraft, 2017; Jackson, 2018; Rose et al., 2022). But understanding which teaching skills contribute to these outcomes is an important empirical question given that teaching effectiveness is multidimensional. Several scholars have also identified school climate as a potential component of culturally responsive teaching (Banks, 2004; Brookover et al., 1978; Ferguson, 2016; Thapa et al., 2013). This is consistent with research documenting disparities in students' experiences with the learning environment. For instance, researchers have found that teachers and administrators disproportionately subject students of color to discipline (Barrett et al., 2021; Liu et al., 2022; Shi & Zhu, 2021b). Others have found that teachers have lower expectations for Black students, on average, and that Black students consequently have less access to advanced coursework (Ferguson, 2003; Hart, 2020; Papageorge et al., 2021; Shi & Zhu, 2021a). These findings suggest that improvements in the school climate may have benefits for students of color. In this paper, we use student surveys conducted by the Massachusetts Department of Elementary and Secondary Education (DESE) beginning in 2018. The Views of Climate and Learning (VOCAL) survey yields scores on three dimensions (engagement, safety, and environment) and nine topics (cultural and linguistic competency, relationships, participation, emotional safety, physical safety, bullying, instruction, mental health, and discipline) of school climate. Although the survey is intended to measure school climate, the survey asks questions specifically about classroom activities and students' relationships with teachers. The survey includes one topic explicitly aligned with the instructional environment, which asks about how teachers support student learning and students' interest in the curriculum as well as whether teachers have high expectations for all students. Other questions, particularly in the participation topic, ask about how well teachers encourage engagement in school. And the relationships and emotional safety topics both include questions about the strength of student-teacher relationships. Finally, one topic specifically intends to measure cultural competence, with questions about demographic representation in instructional materials; the extent to which school personnel respect students regardless of their backgrounds; expectations for advanced coursework; general respect from school personnel; and inclusive social environments. We first find that perceptions of school climate are not uniform for all students. Students of color have significantly lower perceptions of the cultural competency of teachers and other school faculty. These differences are most pronounced for Black students, particularly in middle and high school, and appear even when we compare students within the same classroom. Black students additionally report worse experiences on topics aligned to student-teacher relationships, bullying, and mental health outcomes. Similar findings have been reported in the extensive literature on school climate (Thapa et al., 2013). We then use standard statistical approaches to isolate teacher effects, which we refer to as climate value added (VA). We investigate how climate VA contributes to other student outcomes and explore how the impacts of teachers vary by student race and ethnicity. We estimate that the variation in teacher effects on school climate is about 0.1 standard deviations, which is comparable in magnitude to teacher effects on tests and behavioral measures (Hanushek & Rivkin, 2010; Jackson, 2018). Climate VA is also predictive of student survey responses in out-of-sample analyses that examine differences in teacher assignments within schools. It is also positively correlated with teacher contributions to test scores, and to a lesser extent, an index of several nontest measures. We further document that students' assessments of climate, as shaped by their teachers, differ by student race/ethnicity. Teachers who improve school climate tend to do so for all students, but the perceptions of white students and students of color do differ for individual teachers. We estimate climate VA separately by student racial/ethnic groups and find the correlation between climate VA obtained from white students and students of color is about 0.7. The perceptions of students of color differ most significantly on topics aligned with cultural competency, participation in school, and comfort approaching teachers about emotional or personal problems. We also find evidence of student-teacher racial-matching effects on school climate. Matches improve school climate perceptions by about 0.06 SDs for Black students, which is about 50% of the discrepancy in climate reports between white and Black students. Teachers who contribute positively to school climate also improve academic achievement. We estimate that a one SD improvement in climate VA increases student test scores by about 0.02 SDs for both white students and students of color. When we estimate climate VA separately by student racial/ethnic groups, we find that an inclusive climate--as measured by the portion of a teacher's contribution to school climate that accrues specifically to students of color--has similar effects on student achievement, but only for students of color. A one SD increase in the inclusive climate measure is also associated with an improvement of about 0.02 SDs in test scores for students of color. Our findings provide additional evidence for culturally responsive teaching as a distinct dimension of effective teaching and provide some new evidence on specific mechanisms. Taken together, our findings suggest that teachers contribute to the school climate, that students respond to these contributions, and that they have consequences for student achievement. We also find that these skills overlap with culturally responsive instruction. Teachers' ability to engage students in coursework, hold high expectations for all students, and form supportive relationships contribute to academic achievement for students of color.

Background/Context: Taking even one Advanced Placement (AP), International Baccalaureate (IB), dual enrollment, or other college-preparatory course is positively associated with student outcomes including test scores; high school graduation; and college matriculation, major choice, and completion (Adelman, 2006; An & Taylor, 2019; Author, 2017, 2020; Avery et al., 2018; Conger et al., 2021; Jackson, 2014). Benefits are especially large for students from historically underrepresented backgrounds (An, 2013a, 2013b; Lee et al., 2022; Long et al., 2012). However, large inequities in advanced course participation persist, despite high postsecondary aspirations among students of color and students from low-income backgrounds (Schneider & Saw, 2016). These inequities are substantially driven by within-school inequities in advanced course enrollments (Kolluri, 2018; Patrick et al., 2020; Xu et al., 2021). Purpose/Research Question: We examine whether an automatic enrollment policy in Washington state increases access to advanced courses for students who have historically been underrepresented in these courses. Setting: Academic Acceleration was first implemented in 2010-11 by an urban district serving about 20,000 students south of Seattle. Two years later, the Washington state legislature passed a law encouraging all districts to adopt the policy. Between 2013-14 and 2015-16, 72 of Washington's 263 districts with at least one high school adopted the policy. In 2019, the legislature updated the policy and mandated that all districts adopt Academic Acceleration by 2021-22. Intervention/Program/Practice: Academic Acceleration attempts to increase equity in advanced course enrollment in two ways. First, it requires high schools to automatically identify students as eligible for advanced courses if they met the state standard on statewide assessments, rather than relying on teacher, counselor, student, or parent input. Second, it makes advanced courses opt out rather than opt in for qualified students by requiring schools to automatically enroll all qualified students into the next highest level of coursework in the same or related subject(s). Research Design: We employed a difference-in-differences model to examine changes in course enrollment in policy-implementing districts compared to changes in comparison districts. We define advanced courses as AP, IB, Cambridge International, Running Start (a dual enrollment program at community colleges), or College in the High School (a dual credit program in high schools). We also examined GPA, enrollment in a subsequent advanced course in grade 12 (for grade 11 students), and high school graduation. We estimated separate models by advanced ELA or social studies courses, advanced mathematics courses, and any advanced course regardless of subject area for students in grades 11 and 12. We estimated differential outcomes for qualified students and students in the URM and FRPL groups using two- and three-way interactions. We used linear probability models, implemented two-way fixed effects, and clustered standard errors at the district level (the level of policy variation). Data Collection and Analysis: We used administrative, student-level data from the Washington Office of Superintendent of Public Instruction for all 11th- and 12th-grade students in Washington public schools from 2010-11 through 2018-19. 263 Washington district have at least one high school. We excluded two very early adopting districts and 15 late adopting districts for insufficient pre- or post-policy data. We also excluded six districts with no pre-period high school enrollment. We generated a matched analytic sample of 72 districts in our three implementation cohorts and 168 comparison districts that did not adopt Academic Acceleration before the 2019 legislative update. The prepolicy implementation trends fail the parallel assumptions test for the 2015 and 2016 cohorts. There is an upward trend in the rate of advanced course taking in treatment districts before implementation but not comparison districts, and the formal test (De Chaisemartin & d'Haultfoeuille, 2020) rejects the null hypothesis of parallel trends (p = 0.03). Propensity score matching did not resolve the violations As a result, our findings should be viewed as descriptive, not causal. Findings/Results: Academic Acceleration was associated with a 5.3 percentage point increase in the probability of enrollment in any advanced course for all students, regardless of qualified status, and a 4.8 percentage point increase in the probability of advanced mathematics courses enrollment for qualified students. It also was associated with a 5.1 percentage point increase in qualified students' chances of taking any advanced course in the 2017 cohort. In most cases, Academic Acceleration was associated with a similar change in the odds of advanced course enrollment for qualified URM or FRPL-eligible students and qualified non-URM or non-FRPL-eligible students. In the 2017 cohort, however, qualified non-URM and non-FRPL-eligible students in policy-implementing districts experienced 7.4 and 6.5 percentage point larger increases in the probability of taking advanced courses, respectively, than qualified URM and FRPL-eligible students. Across cohorts, policy implementation was associated with a 2 percentage point larger increase in ELA/social studies enrollment, and with a 3.1 and 3.8 percentage point larger increase in mathematics enrollment, for URM and FRPL-eligible students, respectively, compared with non-URM and non-FRPL-eligible students. Taken together, these changes translate into narrowed enrollment differences by URM and FRPL status. Notably, treated districts experienced increases in advanced course enrollment for URM groups relative to non-URM groups regardless of qualified status. Students' performance in courses targeted by the policy and other courses remained similar or showed small increases from pre- to post-policy. The policy also was associated with a 4.6 percentage-point higher probability of enrolling in another advanced math course in 12th grade (for 11th graders), and no difference in the probability of on-time high school graduation. Conclusions: Increases in the probability of advanced mathematics course enrollment among qualified students in treatment districts suggest that Academic Acceleration may have identified and enrolled qualified students who would not have enrolled before policy implementation. Interestingly, the increases in any advanced course enrollment for all students (qualified and not qualified) suggest that districts with Academic Acceleration policies also may have broadened participation by expanding their eligibility criteria beyond standardized test scores or encouraging students to try an advanced course outside the core subjects. Findings beyond enrollment indicate the policy increased access to advanced courses without decreasing student performance. However, demographic inequities remained, especially in advanced mathematics.