Search

While current higher education literature stresses the importance of equity, diversity, and inclusivity, these imperatives have been mainly absent from conversations related to boards of higher education. In this paper, the authors present a historical overview of the demographic landscape of trustee boards from inception to the present. Using critical literacy as a methodology, the authors problematize the lack of discourses regarding Board's diversity. The authors juxtapose the longstanding homogeneity of boards with the increasing heterogeneity of higher education students and argue that systemic forms of racism have denied the opportunity to diversify those in charge of making decisions in higher education. Additionally, using the case of California, the authors problematize how diversity gaps in board composition manifest even within one of the most diverse and liberal states in the country. Ultimately, the authors make a case for diversifying the board of trustees as an instrumental step to align with the national push for enhanced diversity and equity in higher education.

The significant decrease in student achievement levels following the pandemic has become a pressing national problem, and the Commonwealth of Massachusetts showed some of the sharpest academic achievement declines in the country. To assist schools in recovering from the pandemic, the federal government allocated three waves of funding through its Elementary and Secondary School Emergency Relief (ESSER) package. Massachusetts received over $2.9 billion in ESSER funding that districts have largely allocated for academic remediation programs, such as high-dosage tutoring, summer learning programs, and after-school remedial instruction. This paper reviews these short-term learning loss recovery interventions. Evidence from this review indicates that while these popular learning loss interventions are underpinned by a strong research base, they are difficult to extend to all students and may be unsustainable after one-time federal relief funding is exhausted. To consider long-run policy responses, this paper examines differentiated teacher compensation, permanently extended instructional time, family engagement programs, and college, career, and technical education initiatives as potential strategies for sustaining student success in Massachusetts. Research suggests that if well-designed, these approaches hold promise for not only supporting learning loss recovery efforts but also creating a foundation for continued academic growth over the long run.

The COVID-19 pandemic forced widespread school closures and a shift to remote learning. A growing body of research has examined the effects of remote learning on student outcomes. But the accuracy of the school modality measures used in these studies is questionable. The most common measures--based on self-reports or district website information--are often inconsistent and lack nationwide coverage. Some studies have used cell phone mobility data to identify school modalities, but there is no consensus yet on how to translate device pings into modality measures. This paper contributes to the literature on modality measurement by examining the relationship between mobile device signals and school modality prior to the pandemic and applies those findings to the pandemic period in Michigan and Washington. We compare our results to state-provided closure data and other nationwide sources, including the Return to Learn Tracker and the COVID-19 School Data Hub. Our findings indicate that cell phone mobility data can accurately predict school modality under normal conditions, but the accuracy drops during the pandemic. These results have implications for future research on educational and health outcomes during both pandemic and non-pandemic-related school closures.

This paper examines the long-term impact of the 1993 Massachusetts Education Reform Act (MERA) and the Massachusetts Comprehensive Assessment System (MCAS) on postsecondary education outcomes, with a focus on historically underserved students. MERA aimed to improve educational standards and close achievement gaps through the introduction of MCAS, a statewide assessment system, and the high school Competency Determination (CD) requirement for graduation. More than two decades later, questions remain about how these reforms have influenced students' readiness for and success in postsecondary education. This analysis addresses three key areas: (1) Changes in 10th-grade MCAS performance over time; (2) Student participation and success in key college readiness benchmarks such as advanced coursework, SAT, and high school graduation; and (3) Postsecondary enrollment and degree completion rates.

This paper reviews overall student performance as well as the performance of student subgroups on the assessment system developed in response to the Massachusetts Education Reform Act of 1993 (MERA), the Massachusetts Comprehensive Assessment System (MCAS). Comparing students in Massachusetts to students in the rest of the United States or against students in other countries can not only confirm the rigor of the MCAS, but the comparison can also provide meaning to MCAS scores and ensure that they accurately measure student performance. There are two primary international exams given at regular intervals: (1) the Trends in International Math and Science Study (TIMSS); and (2) the Program for International Student Assessment (PISA). The strong performance on the international exams across several years and subjects, especially on TIMSS, confirmed the quality of Massachusetts K-12 schools.

Education resources matter when they are allocated and used effectively. Yet, the upstream decisions school boards make about district budgets and resource allocation are understudied. In this descriptive study, we analyze data from 400 publicly available video recordings of financial deliberations in school board budget meetings between spring 2022 and spring 2023. Half of the video recordings are from school boards that received education finance training from the Edunomics Lab at Georgetown University. We find school boards discussed student outcomes in only 15-25% of board meetings focused on financial deliberations. Only about 11% of the variation in financial deliberations can be explained by district characteristics, student achievement, and community characteristics. We find no differences in the discussion of student outcomes for districts with and without the Edunomics training. However, descriptive evidence suggests a positive relationship between the Edunomics training and some summary measures of financial deliberations: the overall level of engagement in budgetary discussions; the likelihood per-unit cost and internal barriers (such as decision-making structure) were mentioned; and the likelihood that the budget was linked to outcomes. These findings underscore the variation in school board deliberations and suggest the potential value of training school board members to influence those deliberations.

The effect of remote learning on student performance has been a frequent topic of research and discussion in the aftermath of the COVID-19 pandemic, yet little is known about the impact of remote instruction on the performance of teachers. This study documents how relative effectiveness of teachers changed when moving from in-person to remote instruction and analyzes the characteristics of teachers associated with greater relative effectiveness during remote instruction. Using matched student/teacher-level data from three large metro-Atlanta school districts, we estimate teacher value-added models to measure the association between teacher characteristics and a teacher's relative contribution to test score growth before and during the period of virtual instruction in the 2020-21 school year. We find evidence of increased variation in overall teacher effectiveness during remote instruction. Results are driven by veteran teachers, who appear relatively more effective in virtual instruction than their less-experienced peers, and by the very best in-person teachers, some of which experience large declines in relative effectiveness when shifting to remote instruction.

Four-day school week (4DSW) schedules are growing rapidly across the U.S., with school districts citing perceived benefits to teacher recruitment and retention and student attendance as motivations for adopting the schedule. This study uses panel data from Colorado, one of the states with the highest prevalence of 4DSWs, to investigate the impacts of the 4DSWs on the percentage of teachers with shortage credentials, teacher attrition rates, and student attendance rates. Utilizing a synthetic control difference-in-differences research design, we find 4DSWs have small negative or statistically insignificant effects on teacher recruitment and retention outcomes and find little variation in these effects by school rurality. Examining student attendance outcomes, we estimate a meaningfully small 0.76 percentage point reduction in attendance rates associated with adopting a 4DSW in non-rural schools (equivalent to 46% of these schools' typical yearly fluctuations in ADA) but do not detect an effect in small rural or non-rural schools. These findings suggest that these purported benefits of 4DSWs are not realized in Colorado, warranting concern about the continued use and expansion of this schedule given prior evidence of its negative average impacts on student achievement.

More than half of states have implemented tuition-free college policies aimed at reducing attendance costs and incentivizing enrollment. We review the academic literature on the design features and impacts of these tuition-free policies, and we analyze an initiative Virginia implemented in 2021 called Get a Skill, Get a Job, Get Ahead (G3), which provides tuition-free community college to students enrolled in eligible associate degree, certificate, and noncredit occupational training programs in five high-demand fields. Our descriptive analysis of G3 from 2016-17 through 2022-23 shows that both institutions and students responded to the tuition-free messaging and eligibility criteria. Specifically, G3-eligible institutional program offerings and student enrollment in such programs both increased by roughly 30% within the first two years of program implementation. While Virginia's tuition-free policy promotes enrollment in targeted occupational programs, overall enrollment effects are partially offset by a 3% enrollment reduction in aid-ineligible transfer-oriented programs. To promote skill development and improve labor market outcomes, policymakers should ensure that programs eligible for tuition-free college include pathways to longer term credentials.

Student service member/veteran (SSM/V) university enrollment has grown exponentially in recent years. In response, many U.S. universities have developed military-focused student services to address navigational and social challenges SSM/Vs face on campus. While research suggests these services are beneficial, few studies have empirically examined how often contemporary SSM/Vs engage with them across universities, how engagement connects to predictors of university success, or how SSM/Vs describe such connections. Using social capital theory, surveys (n=531), and interviews (n=59) of SSM/Vs across four universities, we analyze SSM/V military-focused service engagement levels, correlations between engagement and campus belonging and institutional satisfaction, and SSM/V perspectives on engagement. Findings suggest SSM/Vs very rarely engage in these services. Higher engagement, however, is significantly associated with more campus belonging and institutional satisfaction. Interviewees describe how the moral support military-focused service staff offer while providing reliable administrative assistance, as well as SSM/V-dedicated spaces and community building, foster belonging and satisfaction.

Since the 1990s, there has been a growing demand for evidence-based education policy and practice. This demand stems from concerns that education systems are not meeting the needs of a changing world and that education research lacks rigor. While this demand aims to improve the quality of education, silos between different actors often hinder how evidence informs policymaking. We encourage researchers to use a collaborative research approach by involving multiple education actors in the research process to close the gaps between research, policy, and practice. This paper is the third in a series of three working papers meant to serve as references and conversation starters for policymakers and researchers as they navigate pedagogical reform for education system transformation in their local contexts. Together, the three working papers emphasize the need for more locally driven collaborative research on how the interaction of culture, local education ecosystems, and learning theories--collectively called Invisible Pedagogical Mindsets--influences teachers' pedagogical choices in the classroom. Primarily intended for education researchers, Working Paper III advocates the use of collaborative research approaches to actively include multiple education actors in the research process, foster complementary relationships between actors with different expertise, and make research findings more relevant and responsive to the local education ecosystem. The paper has three parts that discuss the need for flexible research approaches to inform policy given the complexities of education decision-making, the importance of communication and dissemination, and how collaborative research can bridge the gaps between research, policy, and practice. The paper concludes by looking at the ongoing work of the SPARKS project at the Center for Universal Education and how collaborative research can contribute to education systems transformation.

Although global access to schooling has increased over the last several decades, Sustainable Development Goal 4 (SDG 4), which champions inclusive, equitable, quality education, is far from being achieved. Experts predict that if the global community continues to operate education systems in the same way, by 2030, only one in six countries will reach the universal secondary school completion targets, and approximately 300 million students in school will continue to lack basic numeracy and literacy skills. The 2022 United Nations Transforming Education Summit emphasized the urgent need for a complete overhaul of education systems to meet SDG 4 targets. One significant outcome of the summit was a call to improve student learning by transforming teacher classroom practice. This paper is the first in a series of three working papers meant to serve as references and conversation starters for policymakers and researchers as they navigate pedagogical reform for education system transformation in their local contexts. This paper explores various definitions of pedagogies, the lack of consensus on what pedagogy means in practice, and the effects of Invisible Pedagogical Mindsets on pedagogical approaches.

In many low- and medium-income countries (LMICs), student-centered pedagogies are often implicitly or explicitly at the heart of innovative pedagogical reforms. In recent years, there has been a growing emphasis on student-centered pedagogies, which aim to shift power dynamics, increase interaction, and prioritize the needs of learners. Many international agencies, governments, and education experts view these pedagogies as "best practices" or a pedagogical "silver bullet" to improve classroom practice. This paper is the second in a series of three working papers meant to serve as references and conversation starters for policymakers and researchers as they navigate pedagogical reform for education system transformation in their local contexts. Together, the three working papers emphasize the need for more locally driven collaborative research on how the interaction of culture, local education ecosystems, and learning theories--collectively called Invisible Pedagogical Mindsets--influences teachers' pedagogical choices in the classroom. This paper details why the authors recommend policymakers examine Invisible Pedagogical Mindsets in their local context to inform pedagogical reforms. The authors discuss the reasons why generalized "best practices"--namely "student-centered pedagogies" as currently implemented--do not often successfully transfer to new cultures, countries, and contexts and argue that many pedagogical reforms do not adequately consider the Invisible Pedagogical Mindsets embedded in each local context.

This paper presents a rationale for using professional and continuing education (PaCE) units at post-secondary institutions throughout California to design and implement talent-pipelines, research and development collaborations, and other knowledge ecosystems where emerging and returning professionals can acquire the knowledge, skills, and abilities (KSAs), as well as the experience, they need to address the challenges of the Fourth Industrial Revolution (4IR). The paper provides an analysis of the reasons why PaCE units are uniquely positioned to address the needs of industry and job seekers, and on a timetable that keeps pace with 4IR velocity.

An ongoing debate in K-12 education policy has been between the "reform" agenda, including charter schools and school vouchers, and advocates of traditional public schools, led by educator unions. A similar split has emerged in higher education, particularly community colleges. Using California as an example, this paper: 1) summarizes the evolution of the current political divide between advocates of the "completion and success" agenda and faculty-led opponents, including the major reforms involved, 2) discusses the claims that leading organizations on each side have made, including their policy priorities, and 3) argues that the two sides share do share some areas of mutual agreement. The paper concludes by noting future policy considerations that could complicate reform efforts.

In recent years, American universities have implemented many innovative strategies to enhance the academic success of students, especially those from underrepresented backgrounds. Yet first-generation and/or low-income (FLI) college students continue to encounter barriers to success because they do not have authentic access to digital technology needed to graduate and be career-ready in our rapidly changing economy. This paper analyzes the current state of digital inequity among FLI students at Stanford University. It also reviews existing programs to address digital inequity at California State University, Fresno (Fresno State), the University of Michigan and Bowdoin College and provides guidance on developing a device program. Finally, the paper recommends strategies to better understand digital inequity and to address it in a sustainable way.

In this paper we examine academic recovery in 12 mid- to large-sized school districts across 10 states during the 2021-22 school year. Our findings highlight the challenges that recovery efforts faced during the 2021-22 school year. Although, on average, math and reading test score gains during the school year reached the pace of pre-pandemic school years, they were not accelerated beyond that pace. This is not surprising given that we found that districts struggled to implement recovery programs at the scale they had planned. In the districts where we had detailed data on student participation in academic interventions, we found that recovery efforts often fell short of original expectations for program scale, intensity of treatment, and impact. Interviews with a subsample of district leaders revealed several implementation challenges, including difficulty engaging targeted students consistently across schools, issues with staffing and limitations to staff capacity, challenges with scheduling, and limited engagement of parents as partners in recovery initiatives. Our findings on the pace and trajectory of recovery and the challenges of implementing recovery initiatives raise important questions about the scale of district recovery efforts. [The working paper received additional funding from Kenneth C. Griffin and AIR Equity Initiative.]

The COVID-19 pandemic devastated student achievement, with declines rivaling those after Hurricane Katrina. These losses widened achievement gaps between historically marginalized students and their peers. Three years later, achievement remains behind pre-pandemic levels for many students. This paper examines 2022-23 academic recovery efforts across eight districts, including tutoring, small group instruction, after-school, extended year, double-dose, digital learning, and expert teacher interventions. Across 22 math and reading interventions, most were delivered to fewer students and for less time than planned. We find positive effects for one tutoring program on math scores and two tutoring programs on reading scores, ranging from 0.22 to 0.33 SD. Each of these programs served a very small share of the district's students and was unlikely to play a major role in district-wide academic recovery. Finally, we find that having an "expert" teacher with high evaluation scores as opposed to a non-expert teacher significantly improves student achievement by 0.06 SD in math and 0.11 SD in reading. While highlighting the promise of intensive academic interventions, our findings underscore the challenges districts face in scaling such interventions to match their recovery needs. The field needs better evidence regarding successful implementation of large-scale interventions.

Asian Americans as a group are overrepresented among STEM college graduates and have the highest average college enrollment rate of any racial or ethnic category. Thus, Asian Americans are typically excluded from educational interventions directed at improving STEM education for Students of Color because they are not considered to be underrepresented minorities. However, statistics obscure the individual needs of the more than 20 ethnic subgroups that fall under the umbrella term Asian Americans. Using a participatory action research approach, this paper documents the institutional and sociocultural factors that push out HMoob (or Hmong) American college students from STEM programs at one large, predominantly White university; and the coordinate processes of gatekeeping and transactional advising that either redirect those students toward non-STEM programs or force them out of the university completely.

Over a 6-year period, a sociolinguistic and sociocultural project was undertaken by Alaska Native expert educators and linguists (aka the Yup'ik Expert Group) from the Yup'ik community in the Lower Kuskokwim School District, Central Alaska. The native experts developed their own culturally sustainable, valid, and reliable Kindergarten through Grade 6 Alaska Native language (Yugtun) assessment. Yup'ik experts named their assessment the Yugtun Piciryaranek Qaneryaranek-llu Cuqyun (aka Yup'ik Culture and Language Measurement). This paper focuses on a foundational component of the assessment, the Yup'ik Cultural Awareness subtest, which has two components, Nonverbal Communication and Yup'ik Worldview.

This paper examines the impact, characteristics, and entrepreneurial proclivities of foreign-born college graduates in the United States. A significant body of research has found that immigrants are more likely to start businesses than those born in the U.S., and the propensity of international students to concentrate in STEM fields indicates enormous potential for economic contributions and innovation. Yet the static nature of the immigration system, with visa pathways and restrictions that discourage business creation, hamper the nation's ability to take full advantage of the benefits immigrants can provide. In fact, this study finds that the U.S. immigration system likely delays foreign-born graduates from creating incorporated firms by as many as five years. The authors estimate that the creation of 150,000 incorporated firms and 580,000 jobs were delayed between 2013 and 2021. Without reform, the U.S. will continue to depress high-value firm creation by international students and cease to be the primary destination of global talent.

The COVID-19 pandemic has presented numerous challenges to students across the United States, particularly those who are parents enrolled in community colleges. California's community college system serves a diverse student population, including a significant number of non-traditional, working adults who are also parents. These students have faced unprecedented challenges due to the pandemic, including the difficulties of balancing childcare responsibilities with academic and professional obligations. This paper summarizes the preliminary findings of a study that intends to contribute to the crucial conversation around childcare needs among community college students. The focus of this study was understanding the experiences of BIPOC (Black, Indigenous, People of Color) mothers with young children and the impact of COVID-19 on their educational experiences in community colleges across the state of California.

In this paper we present the Advancing Racial Equity in Community College Model which maps out the organizational conditions shaping institutional transformation. Focused on two dimensions, the level of "organizational support" and "shared responsibility" to enact equity, we describe four quadrants with distinct organizational conditions that shape how equity advocates design, build, and sustain equity efforts. With well-documented racial inequities and renewed calls for racial justice across higher education, it demands new ways of exploring and understanding how institutional actors leading equity efforts are nested within differing organizational contexts that can enable as well as restrict the enactment and success of racial equity efforts. Our model helps equity advocates gain an "awareness" of known barriers to implementation in higher education, assess the readiness of their campus for racialized change, and take action to build the necessary institutional support and capacity to move the work forward.

In this essay, we explore the consequences of the University of California's policies to address racial disparities and its support for social justice activism as influences on its commitment to academic freedom and other intellectual values. This is a story of the interaction between two essential public university missions -- one civic, the other intellectual -- and the slow effacement of one by the other. The University's expressed commitments to academic freedom and the culture of rationalism have not been abandoned, but they are too often considered secondary or when confronted by new administrative initiatives and social movement activism related to diversity, equity, and inclusion (DEI). The experimental use of mandatory DEI statements on a number of the ten UC campuses, within willing academic departments, as initial screening mechanisms in faculty hiring is the most dramatic of the new administrative policies that have been put into place to advance faculty diversity. This policy can be considered the most problematic of a series of efforts that the UC campuses and the UC Office of the President have taken for more than a decade to prioritize representation in academic appointments. Our intent is to encourage a discussion of these policies within UC in light of the University's fundamental commitments to open intellectual inquiry, the discovery and dissemination of a wide range of new knowledge, and a culture of rationalism.

Professional references are widely used in hiring decisions, yet their effectiveness remains largely understudied. This study analyzes structured ratings collected from the professional references of teacher applicants and conduct an experiment to see whether the ratings influence hiring managers' assessments of applicants and hiring decisions. There is little evidence that providing reference ratings to hiring managers influences their evaluations of candidates or hiring choices in productive ways. However, the analysis suggests that reference ratings are predictive of future job performance independent of other applicant information available to hiring managers. The result is a paradox: reference ratings offer potentially low-cost, high-value information, but hiring managers do not appear to make productive use of them.

This report provides formative data about the implementation of projects associated with the Pennsylvania Bureau of Special Education's Attract, Prepare, and Retain (APR) efforts during the 2023-24 school year. We surveyed or interviewed students and educators participating in six such projects: Developing Future Special Educators Grants, APR Mentoring Project, Networking and Learning Communities, Learning Institutes, Accelerated Programs for PK-12 Special Education Teacher Certification Grant, and American Sign Language programs. For the first three APR projects, these findings build on data from the first year of project implementation in the 2022-23 school year. And for the latter three projects that were introduced in 2023-24, these data provide early evidence about how participants view their experiences with these projects. [This project is funded by a contract with PaTTAN Pittsburgh and the Bureau of Special Education at the Pennsylvania Department of Education.]

Teacher turnover is a perennial concern, especially in low-performing, high-poverty schools. While districts and schools may try to anticipate and mitigate turnover by surveying teachers about their future plans, existing research on whether teacher-reported intent is predictive of actual turnover behavior is mixed. Using unique survey data from teachers in 35 low-performing, high-poverty districts in Michigan linked at the teacher level to statewide administrative data, we are able to measure turnover behavior one, two, and three years following reported intent. We find that intent is a significant predictor of turnover and becomes increasingly predictive over time. We also find organizational commitment and school organizational conditions are important factors in teachers' intent and, to a lesser degree, actual turnover behavior.

This article clarifies historical and sociocultural factors that impact the role of STEM in the racialization of Asian Americans. Drawing on critical race and other theories of Asian American racialization, and a review of empirical research on the experiences of Asian American college students in STEM, we develop a conceptual framework called "STEM Asianization" that highlights the role of STEM ideology in the model minority racialization of Asian Americans. Consequences for Asian American students include (1) erasure of the intersectional experiences of minoritized Asian American students; (2) dehumanization of Asian Americans and establishment of a bamboo ceiling; (3) representation of Asian Americans as a perpetual foreigner/Yellow Peril during times of cultural and political crisis; and (4) representation of Asian Americans who cannot or do not conform to the STEM achievement narrative as a failed minority. We argue that STEM Asianization reproduces White supremacy by ideologically reinforcing the colorblind meritocracy of STEM institutions in the United States. [Additional funding provided by the Wisconsin Louis Stokes Alliance for Minority Participation.]

We study the effect of paternity leave on early child development. We collect sur-vey data on 5,000 children under age six in Spain and exploit several extensions of paternity leave that took place between 2017 and 2021. We follow a differences-in-discontinuities research design, based on the date of birth of each child and using cohorts born in non-reform years as controls. We show that the extensions led to significant increases in the length of leave taken by fathers, without affecting that of mothers, thus increasing parental time at home in the first year after birth. Eligibility for four additional weeks of paternity leave led to a significant 12 percentage-point increase in the fraction of children with developmental delays. We provide evidence for two potential mechanisms. First, children exposed to longer paternity leave spend less time alone with their mother, and more time with their father, during their first year of life. Second, treated children use less formal childcare. Our results suggest that paternity leave replaces higher-quality modes of early care. We conclude that the effects of parental leave policies on children depend crucially on the quality of parental versus counterfactual modes of childcare. [Funding for this report was provided by the Spanish Agencia Estatal de Investigación (AEI) and the Spanish Ministry of Science, Innovation and Universities.]

This study explores the underrepresentation of Black and Latinx students in California's community college Dual Enrollment (DE) programs. The study investigates how DE staff describe an understanding and commitment towards equity for Black and Latinx students in DE programs and how staff engage in equitably aimed praxis to serve Black and Latinx students through practices and collaborations between feeder high schools. Using a Critical Policy Analysis lens, the research highlights how Black and Latinx students are prioritized through equitable practices focused in advising and outreach. However, not all DE staff prioritize Black and Latinx through practices. Despite this, scant instances reveal that collaborative efforts between DE programs, high schools, and districts improve DE services and outcomes, though majority of K-12 partners are absent from collaborative efforts led by DE programs. The study emphasizes the need for increased collaboration between K-12 partners and integrating equitable approaches to DE outreach and advising to engage and recruit Black and Latinx students. This research advances the conversation of equity in DE programs and offers insights for addressing participation gaps among Black and Latinx students.

We estimate the effects of federal pandemic-relief funding (ESSER III) for K12 schools on district-level student achievement growth in 2023. We rely on student test achievement data from over 5,000 school districts across 30 states. Our novel identification strategy exploits variation in ESSER attributable to its allocation rules and their relationship to Title I. We find that each $1,000 increase in ESSER per pupil funds led to statistically significant increases in district math scores of 0.008 standard deviations and similar but statistically insignificant increases in ELA scores. Our heterogeneity analysis suggests impacts were not even across district pre-pandemic spending levels, student race, or urbanicity. Our estimates provide some insight into how much investment may be needed for a full academic recovery from the pandemic: to recover losses remaining after 2023, we estimate schools would need to spend $9,000 to $13,000 per pupil.

We study the effect of educational attainment on family formation using regression discontinuity designs generated by centralized admissions processes to both secondary and tertiary education in Finland. Admission to further education at either margin does not increase the likelihood that men form families. In contrast, women admitted to further education are more likely to both live with a partner and have children. We then pre-register and test two hypotheses which could explain each set of results using survey data. These suggest that the positive association between men's education and family formation observed in the data is driven by selection. For women, our estimates are consistent with the idea that, as increased returns to social skills shift the burden of child development from schools to parents and particularly mothers, education can make women more attractive as potential partners. [Funding for this report was provided by The Strategic Research Council, the Research Council of Finland, and Palkansaajasäätiö.]

Most states responded to the onset of the pandemic by temporarily granting teachers Emergency licenses. These licenses allowed teachers to work in classrooms without passing the typical licensure exams. Since then, several states have extended their use of Emergency licenses, raising questions about how these policies impact the composition of the teacher workforce and student outcomes. In this paper, we examine the result of these policies using data on multiple cohorts of Emergency licensed teachers (ELTs) who taught in Massachusetts between 2021 and 2023. We find that ELTs were slightly more likely to remain in the same school and in the teaching workforce than teachers from other entry routes. However, ELTs' students scored significantly lower on standardized tests in math and science than other students in the same school and same year. Our findings are at odds with earlier, more positive assessments of Emergency licensure in Massachusetts. Our updated results appear to be driven by more recent cohorts of ELTs, rather than the teachers who received Emergency licenses at the start of the pandemic. Overall, this study suggests policymakers should be cautious when drawing sweeping conclusions about the impacts of teacher licensure based solely on the earliest cohort of teachers who obtained pandemic-era Emergency licenses.

The guided pathways model, comprising 14 different practices, is a framework for comprehensive, whole-college reform undertaken by community colleges to help all students choose, enter, progress through, and complete a program of study that enables them to secure sustaining-wage employment or transfer with junior standing in a major. Since its introduction in 2015, it has been adopted by hundreds of community colleges across the United States. This paper asks whether guided pathways practices implemented at 62 community and technical colleges in three states--Tennessee, Ohio, and Washington--are associated with improvements in student outcomes during the first year of college. Specifically, using institutional survey and rich administrative data, we construct measures of adoption of guided pathways reforms to examine the association between guided pathways practices and fall-to-fall persistence, college credits earned, college math credits earned, and STEM credits earned. Our study reveals substantial variation in the adoption of guided pathways reforms across the states and across community colleges within the states over time. While we cannot establish a causal relationship between guided pathways adoption and student outcomes, we find significant positive associations between the statewide adoption of guided pathways reforms and early student outcomes in Tennessee. The observed improvements in that state are likely the result of concurrent reforms--guided pathways and others--implemented simultaneously, rather than of guided pathways reforms alone. We do not find evidence of improved student outcomes in either Ohio or Washington following the launch of statewide guided pathways initiatives. Our findings suggest that complementarities among adopted practices within and across areas of practice--rather than the adoption of individual practices or the intensity of adoption--seem to drive larger improvements in early academic success across the three states. Our study is the first of its kind to explore the potential of guided pathways reforms in contributing to improved early academic success, representing a significant descriptive contribution given that whole-college reforms in higher education are understudied.

Novelty evaluation is vital for the promotion and management of innovation. With the advancement of information techniques and the open data movement, some progress has been made in novelty measurements. Tracking and reviewing novelty measures provides a data-driven way to assess contributions, progress, and emerging directions in the science field. As academic papers serve as the primary medium for the dissemination, validation, and discussion of scientific knowledge, this review aims to offer a systematic analysis of novelty measurements for scientific papers. We began by comparing the differences between scientific novelty and four similar concepts, including originality, scientific innovation, creativity, and scientific breakthrough. Next, we reviewed the types of scientific novelty. Then, we classified existing novelty measures according to data types and reviewed the measures for each type. Subsequently, we surveyed the approaches employed in validating novelty measures and examined the current tools and datasets associated with these measures. Finally, we proposed several open issues for future studies.

LLMs are increasingly being used to assess the relevance of information objects. This work reports on experiments to study the labelling of short texts (i.e., passages) for relevance, using multiple open-source and proprietary LLMs. While the overall agreement of some LLMs with human judgements is comparable to human-to-human agreement measured in previous research, LLMs are more likely to label passages as relevant compared to human judges, indicating that LLM labels denoting non-relevance are more reliable than those indicating relevance. This observation prompts us to further examine cases where human judges and LLMs disagree, particularly when the human judge labels the passage as non-relevant and the LLM labels it as relevant. Results show a tendency for many LLMs to label passages that include the original query terms as relevant. We, therefore, conduct experiments to inject query words into random and irrelevant passages, not unlike the way we inserted the query "best caf\'e near me" into this paper. The results show that LLMs are highly influenced by the presence of query words in the passages under assessment, even if the wider passage has no relevance to the query. This tendency of LLMs to be fooled by the mere presence of query words demonstrates a weakness in our current measures of LLM labelling: relying on overall agreement misses important patterns of failures. There is a real risk of bias in LLM-generated relevance labels and, therefore, a risk of bias in rankers trained on those labels. We also investigate the effects of deliberately manipulating LLMs by instructing them to label passages as relevant, similar to the instruction "this paper is perfectly relevant" inserted above. We find that such manipulation influences the performance of some LLMs, highlighting the critical need to consider potential vulnerabilities when deploying LLMs in real-world applications., Comment: Published in the proceedings of SIGIR-AP'24

This study proposes a data-driven framework for enhancing the accuracy and efficiency of scientific peer review through an open, bottom-up process that estimates reviewer quality. Traditional closed peer review systems, while essential for quality control, are often slow, costly, and subject to biases that can impede scientific progress. Here, we introduce a method that evaluates individual reviewer reliability by quantifying agreement with community consensus scores and applying Bayesian weighting to refine paper quality assessments. We analyze open peer review data from two major scientific conferences, and demonstrate that reviewer-specific quality scores significantly improve the reliability of paper quality estimation. Perhaps surprisingly, we find that reviewer quality scores are unrelated to authorship quality. Our model incorporates incentive structures to recognize high-quality reviewers and encourage broader coverage of submitted papers, thereby mitigating the common "rich-get-richer" pitfall of social media. These findings suggest that open peer review, with mechanisms for estimating and incentivizing reviewer quality, offers a scalable and equitable alternative for scientific publishing, with potential to enhance the speed, fairness, and transparency of the peer review process., Comment: 15 pages, 5 main text figures, 3 supplementary figures

We introduce PaSa, an advanced Paper Search agent powered by large language models. PaSa can autonomously make a series of decisions, including invoking search tools, reading papers, and selecting relevant references, to ultimately obtain comprehensive and accurate results for complex scholarly queries. We optimize PaSa using reinforcement learning with a synthetic dataset, AutoScholarQuery, which includes 35k fine-grained academic queries and corresponding papers sourced from top-tier AI conference publications. Additionally, we develop RealScholarQuery, a benchmark collecting real-world academic queries to assess PaSa performance in more realistic scenarios. Despite being trained on synthetic data, PaSa significantly outperforms existing baselines on RealScholarQuery, including Google, Google Scholar, Google with GPT-4 for paraphrased queries, chatGPT (search-enabled GPT-4o), GPT-o1, and PaSa-GPT-4o (PaSa implemented by prompting GPT-4o). Notably, PaSa-7B surpasses the best Google-based baseline, Google with GPT-4o, by 37.78% in recall@20 and 39.90% in recall@50. It also exceeds PaSa-GPT-4o by 30.36% in recall and 4.25% in precision. Model, datasets, and code are available at https://github.com/bytedance/pasa.

Large language models (LLMs) have significantly impacted human society, influencing various domains. Among them, academia is not simply a domain affected by LLMs, but it is also the pivotal force in the development of LLMs. In academic publications, this phenomenon is represented during the incorporation of LLMs into the peer review mechanism for reviewing manuscripts. We proposed the concept of automated scholarly paper review (ASPR) in our previous paper. As the incorporation grows, it now enters the coexistence phase of ASPR and peer review, which is described in that paper. LLMs hold transformative potential for the full-scale implementation of ASPR, but they also pose new issues and challenges that need to be addressed. In this survey paper, we aim to provide a holistic view of ASPR in the era of LLMs. We begin with a survey to find out which LLMs are used to conduct ASPR. Then, we review what ASPR-related technological bottlenecks have been solved with the incorporation of LLM technology. After that, we move on to explore new methods, new datasets, new source code, and new online systems that come with LLMs for ASPR. Furthermore, we summarize the performance and issues of LLMs in ASPR, and investigate the attitudes and reactions of publishers and academia to ASPR. Lastly, we discuss the challenges associated with the development of LLMs for ASPR. We hope this survey can serve as an inspirational reference for the researchers and promote the progress of ASPR for its actual implementation., Comment: Work in progress

Figures and their captions play a key role in scientific publications. However, despite their importance, many captions in published papers are poorly crafted, largely due to a lack of attention by paper authors. While prior AI research has explored caption generation, it has mainly focused on reader-centered use cases, where users evaluate generated captions rather than actively integrating them into their writing. This paper addresses this gap by investigating how paper authors incorporate AI-generated captions into their writing process through a user study involving 18 participants. Each participant rewrote captions for two figures from their own recently published work, using captions generated by state-of-the-art AI models as a resource. By analyzing video recordings of the writing process through interaction analysis, we observed that participants often began by copying and refining AI-generated captions. Paper writers favored longer, detail-rich captions that integrated textual and visual elements but found current AI models less effective for complex figures. These findings highlight the nuanced and diverse nature of figure caption composition, revealing design opportunities for AI systems to better support the challenges of academic writing., Comment: This paper will appear at AAAI 2025 Workshop (2nd AI4Research Workshop: Towards a Knowledge-grounded Scientific Research Lifecycle)

In tenure decisions, the treatment of co-authored papers often raises questions about a candidate's research independence. This study examines the effects of solo versus collaborative authorship in high-profile Economics journals on long-term academic success. Our findings confirms the traditional belief that solo-authored publications significantly enhance long-term research output and citation impact compared to collaborative efforts. However, relative to solo-authored papers, international collaborations have a less negative impact on long-term success than national and institutional collaborations. Temporal trends highlight the increasing importance of diverse and international collaborations. These insights provide actionable guidance for tenure committees on evaluating co-authored work and for researchers on optimizing their publication strategies., Comment: 13 pages, including 1 table and references

We present OpenReviewer, an open-source system for generating high-quality peer reviews of machine learning and AI conference papers. At its core is Llama-OpenReviewer-8B, an 8B parameter language model specifically fine-tuned on 79,000 expert reviews from top ML conferences. Given a PDF paper submission and review template as input, OpenReviewer extracts the full text, including technical content like equations and tables, and generates a structured review following conference-specific guidelines. Our evaluation on 400 test papers shows that OpenReviewer produces significantly more critical and realistic reviews compared to general-purpose LLMs like GPT-4 and Claude-3.5. While other LLMs tend toward overly positive assessments, OpenReviewer's recommendations closely match the distribution of human reviewer ratings. The system provides authors with rapid, constructive feedback to improve their manuscripts before submission, though it is not intended to replace human peer review. OpenReviewer is available as an online demo and open-source tool., Comment: Demo: https://huggingface.co/spaces/maxidl/openreviewer Model: https://huggingface.co/maxidl/Llama-OpenReviewer-8B

Peer review, as a cornerstone of scientific research, ensures the integrity and quality of scholarly work by providing authors with objective feedback for refinement. However, in the traditional peer review process, authors often receive vague or insufficiently detailed feedback, which provides limited assistance and leads to a more time-consuming review cycle. If authors can identify some specific weaknesses in their paper, they can not only address the reviewer's concerns but also improve their work. This raises the critical question of how to enhance authors' comprehension of review comments. In this paper, we present SEAGraph, a novel framework developed to clarify review comments by uncovering the underlying intentions behind them. We construct two types of graphs for each paper: the semantic mind graph, which captures the author's thought process, and the hierarchical background graph, which delineates the research domains related to the paper. A retrieval method is then designed to extract relevant content from both graphs, facilitating coherent explanations for the review comments. Extensive experiments show that SEAGraph excels in review comment understanding tasks, offering significant benefits to authors.

Section identification is an important task for library science, especially knowledge management. Identifying the sections of a paper would help filter noise in entity and relation extraction. In this research, we studied the paper section identification problem in the context of Chinese medical literature analysis, where the subjects, methods, and results are more valuable from a physician's perspective. Based on previous studies on English literature section identification, we experiment with the effective features to use with classic machine learning algorithms to tackle the problem. It is found that Conditional Random Fields, which consider sentence interdependency, is more effective in combining different feature sets, such as bag-of-words, part-of-speech, and headings, for Chinese literature section identification. Moreover, we find that classic machine learning algorithms are more effective than generic deep learning models for this problem. Based on these observations, we design a novel deep learning model, the Structural Bidirectional Long Short-Term Memory (SLSTM) model, which models word and sentence interdependency together with the contextual information. Experiments on a human-curated asthma literature dataset show that our approach outperforms the traditional machine learning methods and other deep learning methods and achieves close to 90% precision and recall in the task. The model shows good potential for use in other text mining tasks. The research has significant methodological and practical implications.

Digital libraries provide different access paths, allowing users to explore their collections. For instance, paper recommendation suggests literature similar to some selected paper. Their implementation is often cost-intensive, especially if neural methods are applied. Additionally, it is hard for users to understand or guess why a recommendation should be relevant for them. That is why we tackled the problem from a different perspective. We propose XGPRec, a graph-based and thus explainable method which we integrate into our existing graph-based biomedical discovery system. Moreover, we show that XGPRec (1) can, in terms of computational costs, manage a real digital library collection with 37M documents from the biomedical domain, (2) performs well on established test collections and concept-centric information needs, and (3) generates explanations that proved to be beneficial in a preliminary user study. We share our code so that user libraries can build upon XGPRec., Comment: Technical Report of our accepted paper at AI4LAC@JCDL2024. 12 pages, 3 figures