Search

While current higher education literature stresses the importance of equity, diversity, and inclusivity, these imperatives have been mainly absent from conversations related to boards of higher education. In this paper, the authors present a historical overview of the demographic landscape of trustee boards from inception to the present. Using critical literacy as a methodology, the authors problematize the lack of discourses regarding Board's diversity. The authors juxtapose the longstanding homogeneity of boards with the increasing heterogeneity of higher education students and argue that systemic forms of racism have denied the opportunity to diversify those in charge of making decisions in higher education. Additionally, using the case of California, the authors problematize how diversity gaps in board composition manifest even within one of the most diverse and liberal states in the country. Ultimately, the authors make a case for diversifying the board of trustees as an instrumental step to align with the national push for enhanced diversity and equity in higher education.

In many low- and medium-income countries (LMICs), student-centered pedagogies are often implicitly or explicitly at the heart of innovative pedagogical reforms. In recent years, there has been a growing emphasis on student-centered pedagogies, which aim to shift power dynamics, increase interaction, and prioritize the needs of learners. Many international agencies, governments, and education experts view these pedagogies as "best practices" or a pedagogical "silver bullet" to improve classroom practice. This paper is the second in a series of three working papers meant to serve as references and conversation starters for policymakers and researchers as they navigate pedagogical reform for education system transformation in their local contexts. Together, the three working papers emphasize the need for more locally driven collaborative research on how the interaction of culture, local education ecosystems, and learning theories--collectively called Invisible Pedagogical Mindsets--influences teachers' pedagogical choices in the classroom. This paper details why the authors recommend policymakers examine Invisible Pedagogical Mindsets in their local context to inform pedagogical reforms. The authors discuss the reasons why generalized "best practices"--namely "student-centered pedagogies" as currently implemented--do not often successfully transfer to new cultures, countries, and contexts and argue that many pedagogical reforms do not adequately consider the Invisible Pedagogical Mindsets embedded in each local context.

Although global access to schooling has increased over the last several decades, Sustainable Development Goal 4 (SDG 4), which champions inclusive, equitable, quality education, is far from being achieved. Experts predict that if the global community continues to operate education systems in the same way, by 2030, only one in six countries will reach the universal secondary school completion targets, and approximately 300 million students in school will continue to lack basic numeracy and literacy skills. The 2022 United Nations Transforming Education Summit emphasized the urgent need for a complete overhaul of education systems to meet SDG 4 targets. One significant outcome of the summit was a call to improve student learning by transforming teacher classroom practice. This paper is the first in a series of three working papers meant to serve as references and conversation starters for policymakers and researchers as they navigate pedagogical reform for education system transformation in their local contexts. This paper explores various definitions of pedagogies, the lack of consensus on what pedagogy means in practice, and the effects of Invisible Pedagogical Mindsets on pedagogical approaches.

Since the 1990s, there has been a growing demand for evidence-based education policy and practice. This demand stems from concerns that education systems are not meeting the needs of a changing world and that education research lacks rigor. While this demand aims to improve the quality of education, silos between different actors often hinder how evidence informs policymaking. We encourage researchers to use a collaborative research approach by involving multiple education actors in the research process to close the gaps between research, policy, and practice. This paper is the third in a series of three working papers meant to serve as references and conversation starters for policymakers and researchers as they navigate pedagogical reform for education system transformation in their local contexts. Together, the three working papers emphasize the need for more locally driven collaborative research on how the interaction of culture, local education ecosystems, and learning theories--collectively called Invisible Pedagogical Mindsets--influences teachers' pedagogical choices in the classroom. Primarily intended for education researchers, Working Paper III advocates the use of collaborative research approaches to actively include multiple education actors in the research process, foster complementary relationships between actors with different expertise, and make research findings more relevant and responsive to the local education ecosystem. The paper has three parts that discuss the need for flexible research approaches to inform policy given the complexities of education decision-making, the importance of communication and dissemination, and how collaborative research can bridge the gaps between research, policy, and practice. The paper concludes by looking at the ongoing work of the SPARKS project at the Center for Universal Education and how collaborative research can contribute to education systems transformation.

This paper examines the impact, characteristics, and entrepreneurial proclivities of foreign-born college graduates in the United States. A significant body of research has found that immigrants are more likely to start businesses than those born in the U.S., and the propensity of international students to concentrate in STEM fields indicates enormous potential for economic contributions and innovation. Yet the static nature of the immigration system, with visa pathways and restrictions that discourage business creation, hamper the nation's ability to take full advantage of the benefits immigrants can provide. In fact, this study finds that the U.S. immigration system likely delays foreign-born graduates from creating incorporated firms by as many as five years. The authors estimate that the creation of 150,000 incorporated firms and 580,000 jobs were delayed between 2013 and 2021. Without reform, the U.S. will continue to depress high-value firm creation by international students and cease to be the primary destination of global talent.

We study the effect of paternity leave on early child development. We collect sur-vey data on 5,000 children under age six in Spain and exploit several extensions of paternity leave that took place between 2017 and 2021. We follow a differences-in-discontinuities research design, based on the date of birth of each child and using cohorts born in non-reform years as controls. We show that the extensions led to significant increases in the length of leave taken by fathers, without affecting that of mothers, thus increasing parental time at home in the first year after birth. Eligibility for four additional weeks of paternity leave led to a significant 12 percentage-point increase in the fraction of children with developmental delays. We provide evidence for two potential mechanisms. First, children exposed to longer paternity leave spend less time alone with their mother, and more time with their father, during their first year of life. Second, treated children use less formal childcare. Our results suggest that paternity leave replaces higher-quality modes of early care. We conclude that the effects of parental leave policies on children depend crucially on the quality of parental versus counterfactual modes of childcare. [Funding for this report was provided by the Spanish Agencia Estatal de Investigación (AEI) and the Spanish Ministry of Science, Innovation and Universities.]

In this essay, we explore the consequences of the University of California's policies to address racial disparities and its support for social justice activism as influences on its commitment to academic freedom and other intellectual values. This is a story of the interaction between two essential public university missions -- one civic, the other intellectual -- and the slow effacement of one by the other. The University's expressed commitments to academic freedom and the culture of rationalism have not been abandoned, but they are too often considered secondary or when confronted by new administrative initiatives and social movement activism related to diversity, equity, and inclusion (DEI). The experimental use of mandatory DEI statements on a number of the ten UC campuses, within willing academic departments, as initial screening mechanisms in faculty hiring is the most dramatic of the new administrative policies that have been put into place to advance faculty diversity. This policy can be considered the most problematic of a series of efforts that the UC campuses and the UC Office of the President have taken for more than a decade to prioritize representation in academic appointments. Our intent is to encourage a discussion of these policies within UC in light of the University's fundamental commitments to open intellectual inquiry, the discovery and dissemination of a wide range of new knowledge, and a culture of rationalism.

This paper presents a rationale for using professional and continuing education (PaCE) units at post-secondary institutions throughout California to design and implement talent-pipelines, research and development collaborations, and other knowledge ecosystems where emerging and returning professionals can acquire the knowledge, skills, and abilities (KSAs), as well as the experience, they need to address the challenges of the Fourth Industrial Revolution (4IR). The paper provides an analysis of the reasons why PaCE units are uniquely positioned to address the needs of industry and job seekers, and on a timetable that keeps pace with 4IR velocity.

An ongoing debate in K-12 education policy has been between the "reform" agenda, including charter schools and school vouchers, and advocates of traditional public schools, led by educator unions. A similar split has emerged in higher education, particularly community colleges. Using California as an example, this paper: 1) summarizes the evolution of the current political divide between advocates of the "completion and success" agenda and faculty-led opponents, including the major reforms involved, 2) discusses the claims that leading organizations on each side have made, including their policy priorities, and 3) argues that the two sides share do share some areas of mutual agreement. The paper concludes by noting future policy considerations that could complicate reform efforts.

In recent years, American universities have implemented many innovative strategies to enhance the academic success of students, especially those from underrepresented backgrounds. Yet first-generation and/or low-income (FLI) college students continue to encounter barriers to success because they do not have authentic access to digital technology needed to graduate and be career-ready in our rapidly changing economy. This paper analyzes the current state of digital inequity among FLI students at Stanford University. It also reviews existing programs to address digital inequity at California State University, Fresno (Fresno State), the University of Michigan and Bowdoin College and provides guidance on developing a device program. Finally, the paper recommends strategies to better understand digital inequity and to address it in a sustainable way.

The COVID-19 pandemic has presented numerous challenges to students across the United States, particularly those who are parents enrolled in community colleges. California's community college system serves a diverse student population, including a significant number of non-traditional, working adults who are also parents. These students have faced unprecedented challenges due to the pandemic, including the difficulties of balancing childcare responsibilities with academic and professional obligations. This paper summarizes the preliminary findings of a study that intends to contribute to the crucial conversation around childcare needs among community college students. The focus of this study was understanding the experiences of BIPOC (Black, Indigenous, People of Color) mothers with young children and the impact of COVID-19 on their educational experiences in community colleges across the state of California.

In this paper we present the Advancing Racial Equity in Community College Model which maps out the organizational conditions shaping institutional transformation. Focused on two dimensions, the level of "organizational support" and "shared responsibility" to enact equity, we describe four quadrants with distinct organizational conditions that shape how equity advocates design, build, and sustain equity efforts. With well-documented racial inequities and renewed calls for racial justice across higher education, it demands new ways of exploring and understanding how institutional actors leading equity efforts are nested within differing organizational contexts that can enable as well as restrict the enactment and success of racial equity efforts. Our model helps equity advocates gain an "awareness" of known barriers to implementation in higher education, assess the readiness of their campus for racialized change, and take action to build the necessary institutional support and capacity to move the work forward.

Most states responded to the onset of the pandemic by temporarily granting teachers Emergency licenses. These licenses allowed teachers to work in classrooms without passing the typical licensure exams. Since then, several states have extended their use of Emergency licenses, raising questions about how these policies impact the composition of the teacher workforce and student outcomes. In this paper, we examine the result of these policies using data on multiple cohorts of Emergency licensed teachers (ELTs) who taught in Massachusetts between 2021 and 2023. We find that ELTs were slightly more likely to remain in the same school and in the teaching workforce than teachers from other entry routes. However, ELTs' students scored significantly lower on standardized tests in math and science than other students in the same school and same year. Our findings are at odds with earlier, more positive assessments of Emergency licensure in Massachusetts. Our updated results appear to be driven by more recent cohorts of ELTs, rather than the teachers who received Emergency licenses at the start of the pandemic. Overall, this study suggests policymakers should be cautious when drawing sweeping conclusions about the impacts of teacher licensure based solely on the earliest cohort of teachers who obtained pandemic-era Emergency licenses.

This study explores the underrepresentation of Black and Latinx students in California's community college Dual Enrollment (DE) programs. The study investigates how DE staff describe an understanding and commitment towards equity for Black and Latinx students in DE programs and how staff engage in equitably aimed praxis to serve Black and Latinx students through practices and collaborations between feeder high schools. Using a Critical Policy Analysis lens, the research highlights how Black and Latinx students are prioritized through equitable practices focused in advising and outreach. However, not all DE staff prioritize Black and Latinx through practices. Despite this, scant instances reveal that collaborative efforts between DE programs, high schools, and districts improve DE services and outcomes, though majority of K-12 partners are absent from collaborative efforts led by DE programs. The study emphasizes the need for increased collaboration between K-12 partners and integrating equitable approaches to DE outreach and advising to engage and recruit Black and Latinx students. This research advances the conversation of equity in DE programs and offers insights for addressing participation gaps among Black and Latinx students.

We study the effect of educational attainment on family formation using regression discontinuity designs generated by centralized admissions processes to both secondary and tertiary education in Finland. Admission to further education at either margin does not increase the likelihood that men form families. In contrast, women admitted to further education are more likely to both live with a partner and have children. We then pre-register and test two hypotheses which could explain each set of results using survey data. These suggest that the positive association between men's education and family formation observed in the data is driven by selection. For women, our estimates are consistent with the idea that, as increased returns to social skills shift the burden of child development from schools to parents and particularly mothers, education can make women more attractive as potential partners. [Funding for this report was provided by The Strategic Research Council, the Research Council of Finland, and Palkansaajasäätiö.]

This study used Postsecondary Student Information System (PSIS) administrative data within the Education and Labour Market Longitudinal Platform to compare enrolment and persistence in postsecondary education (PSE) among high school graduates in British Columbia with and without special needs across five cohorts from 2010/2011 to 2014/2015 before and after controlling for several sociodemographic characteristics and academic achievement. The use of integrated longitudinal administrative data from the British Columbia Ministry of Education, the PSIS and the T1 Family File and the disaggregation of the special needs categorization were two major strengths of this study. Results show that high school graduates with mental health-related or cognitive needs and those with physical or sensory needs were less likely to enrol in PSE compared with high school graduates without special needs, even after controlling for covariates. Moreover, graduates with mental health-related or cognitive needs were less likely to transition to PSE immediately and less likely to persist in PSE two years after enrolment. These findings suggest that high school graduates with special needs, particularly those with mental health-related or cognitive needs, may encounter different types of barriers in transitioning to PSE.

Using British Columbia Ministry of Education administrative school data within the Education and Labour Market Longitudinal Platform, this study compared the proportions of high school graduates among Grade 12 students with and without special needs across nine cohorts from 2010/2011 to 2018/2019 before and after controlling for several sociodemographic characteristics. Two major strengths of this study were the use of longitudinal administrative education data integrated with income tax data from the T1 Family File and the further disaggregation of the special education needs categorization. Students with special needs in all different categories (excluding those with gifted status) were less likely to have graduated across all nine cohorts compared with students without special needs, even after controlling for sociodemographic characteristics and academic achievement, suggesting that students with special needs may face other types of barriers in completing high school. Yet there was diversity among students with special needs, with the highest proportions of graduation among students with learning disabilities or those with sensory needs and the lowest among students with intellectual disabilities. A larger share of females than males graduated high school among students without special needs. However, sex differences were less consistent among students with special needs status (including students with gifted status). As expected, the proportions of graduation were significantly higher at age 19 compared with at age 18 or younger, with the differences being slightly higher among students with special needs (excluding those with gifted status; 5 to 10 percentage points) compared with those without special needs (3 to 7 percentage points). The largest age differences were observed among students with autism spectrum disorder, behavioural needs or mental illness, and those with physical needs across all nine cohorts.

The Elementary and Secondary School Emergency Relief Fund (ESSER) was the largest onetime federal investment in K-12 schools in history, funneling almost $200 billion to states and school districts. We use novel data from Washington State to investigate the extent to which ESSER funding causally influenced spending on school personnel. We argue one cannot infer this directly from ESSER claims data because of the fungibility of school budgets. Thus, we rely on a more direct signal of district hiring decisions: public education job postings scraped from district hiring websites. To address endogeneity concerns, our preferred approach employs an instrumental variables strategy that exploits a formula mechanism used to determine Title I funding for 2020-21 (and thus ESSER allocations in 2022) based on the number of Title I formula-eligible children. We find strong, arguably causal, evidence that public school hiring increased in response to the availability of ESSER funding. Specifically, we estimate that each $1,000 in ESSER allocations caused districts to seek to hire $206 in additional staff, disproportionately teachers. These estimates suggest that roughly 12,000 new staff (including 5,100 teachers) were hired in Washington because of ESSER. In the absence of new funding, school staffing budgets will likely need to contract substantially following the sunset of ESSER.

Departmentalized instruction, in which teachers specialize in one or more core subjects and instruct multiple groups of students in a day, has become increasingly prominent in elementary schools. Using 8 years of data from Massachusetts and a difference-in-differences design, we estimate the effects of departmentalization on student achievement. We find that departmentalization has positive effects in English language arts (ELA) and science and mixed evidence of positive effects in math. These positive effects are not driven by teacher productivity improvements: Consistent with prior findings on teacher specialization, teachers are less effective when specializing in math and no more effective in ELA than when teaching self-contained classrooms. Rather, consistent with the theoretical underpinnings for specialization, departmentalized schools tend to assign teachers to their stronger subjects.

The guided pathways model, comprising 14 different practices, is a framework for comprehensive, whole-college reform undertaken by community colleges to help all students choose, enter, progress through, and complete a program of study that enables them to secure sustaining-wage employment or transfer with junior standing in a major. Since its introduction in 2015, it has been adopted by hundreds of community colleges across the United States. This paper asks whether guided pathways practices implemented at 62 community and technical colleges in three states--Tennessee, Ohio, and Washington--are associated with improvements in student outcomes during the first year of college. Specifically, using institutional survey and rich administrative data, we construct measures of adoption of guided pathways reforms to examine the association between guided pathways practices and fall-to-fall persistence, college credits earned, college math credits earned, and STEM credits earned. Our study reveals substantial variation in the adoption of guided pathways reforms across the states and across community colleges within the states over time. While we cannot establish a causal relationship between guided pathways adoption and student outcomes, we find significant positive associations between the statewide adoption of guided pathways reforms and early student outcomes in Tennessee. The observed improvements in that state are likely the result of concurrent reforms--guided pathways and others--implemented simultaneously, rather than of guided pathways reforms alone. We do not find evidence of improved student outcomes in either Ohio or Washington following the launch of statewide guided pathways initiatives. Our findings suggest that complementarities among adopted practices within and across areas of practice--rather than the adoption of individual practices or the intensity of adoption--seem to drive larger improvements in early academic success across the three states. Our study is the first of its kind to explore the potential of guided pathways reforms in contributing to improved early academic success, representing a significant descriptive contribution given that whole-college reforms in higher education are understudied.

Multi-campus public higher education governance systems exist in 44 of the 50 U.S. states. They include all the largest and most influential public colleges and universities in the United States, educating fully 75 percent of the nation's public sector students. Their impact is enormous. And yet, they are largely neglected and as a tool for improvement are underutilized. Meanwhile, many states continue to struggle achieving their goals for higher education attainment, social and economic mobility, workforce development, equitable access and affordability, technological innovation, and human and environmental health. The dearth of scholarly research on these systems and their more effective use is explored in a forthcoming volume edited by the author. This paper extracts from that volume a set of specific ways in which systems can leverage their unique ability to use scale in service to their mission.

In 1993, the Commonwealth of Massachusetts dramatically overhauled its K-12 education system and created a new school finance formula, building an educational accountability structure to ensure every child has access to a high-quality education. The Massachusetts Education Reform Act (MERA) established academic standards in core subjects, mandated assessments to measure student outcomes on those standards, and established a system for holding schools accountable when students failed to meet basic expectations. This system has helped Massachusetts' public schools become the highest performing in the country. Student outcomes in all tested subjects and across demographic groups have improved steadily over time, but disparities in achievement and attainment exist between the Commonwealth's most privileged students and their less privileged counterparts, many of whom are black or Hispanic. Without the MERA and its requirement to assess every student and publish aggregate academic outcomes, policymakers may not understand the extent of disparity or how to address it as student outcomes data are integral to understanding where Massachusetts' public schools have been, where they are going, and how they can get there. This paper illustrates the importance of the Massachusetts Education Reform Act and how it has positively impacted students over time. It explains why the current accountability system evolved as it did and why preserving the most important aspects of that system is critical if the state is going to fulfill its constitutional obligation to educate all children to a high common standard.

It is now well established that the COVID-19 pandemic had a devastating and unequal impact on student achievement. Test score declines were disproportionately large for historically marginalized students, exacerbating preexisting achievement gaps and threatening educational and economic inequality. In this paper, we use longitudinal student-level NWEA MAP Growth test data to estimate differences in test score declines for students at different points on the prepandemic test distribution. We also test the extent to which students' schools and districts accounted for these differences in declines. We find significant differences in learning loss by baseline achievement, with lower-achieving student's scores dropping 0.100 SD more in math and 0.113 SD more in reading than higher-achieving students' scores. We additionally show that the school a student attended accounts for about three-quarters of this widening gap in math achievement and about one-third in reading. The findings suggest school and district-level policies may have mattered more for learning loss than individual students' experiences within schools and districts. Such nuanced information regarding the variation in the pandemic's impacts on students is critical for policymakers and practitioners designing targeted academic interventions and for tracking disparities in academic recovery. [Additional funding for this report was provided by Kenneth C. Griffin.]

This document provides technical detail and supporting data for the research findings discussed in 'The student journey in VET: the many faces of completion and non-completion'. The analysis in this technical paper explores: (1) an approach to identifying VET subject enrolment activity that serves a compliance or regulatory purpose; (2) variability in completion rates across VET qualifications, and associated differences in patterns of subject enrolments and outcomes; (3) different indicators of student outcomes in VET, including program completion, subject completion, and movement to subsequent VET; (4) student training pathways exploring the extent to which students undertook programs, stand-alone subjects, or a combination of the two, and how this training choice evolved over time; and (5) student training pathways exploring the extent to which students went on to enrol in a program at a higher, lower, or the same level of educations, and how these pathways compared for students who did and did not complete their initial program.

Each year, more than a million high school students nationally take college dual enrollment courses, which have been shown to increase college access and success among participants. Yet racial/ethnic and other equity gaps in dual enrollment participation are widespread. To broaden the benefits of dual enrollment, the state of Ohio passed legislation in 2017 establishing the Innovative Programs (IP) policy, allowing waivers to test-based eligibility requirements--a frequently identified barrier to equitable access--for specific high school-college partnerships providing expanded outreach and support for students underrepresented in the state's dual enrollment program. This paper describes a multiple methods study of IP we conducted to examine how these partnerships were implemented to address the needs of underrepresented students and to evaluate whether the partnerships were successful in broadening access to and success in dual enrollment, as measured by course participation, pass rates, and college matriculation after high school. We find that the IP increased participation in dual enrollment among Black and Hispanic students. And while the implementation of the policy broadened access without changing course outcomes, the impacts on college enrollment after high school were mixed. Our results underscore the importance of pairing increased access to dual enrollment with adequate financial, advising, and academic resources to promote student success in and beyond dual enrollment courses.

Who ends up in the teacher workforce is greatly influenced by who is admitted into teacher education programs (TEPs). To better understand how the preferences of teacher education faculty might shape admissions of STEM teacher candidates, we surveyed faculty who teach content or methods courses to STEM teacher candidates across five universities. Faculty reported that they most value information collected from individual interviews with applicants and data on the number of STEM courses taken in college and their performance in these courses, and least value data on university admissions tests, high school GPA, and teacher licensure test scores. When we investigate faculty members' revealed preferences through a conjoint analysis, we find that faculty most value applicants who have worked with students from diverse backgrounds and applicants from a marginalized racial or ethnic community, and least value whether they received high grades in math and/or science courses. Finally, we find significant variation in these perceptions across respondents in different faculty roles, who teach different courses, and from different institutions: for example, Arts and Sciences faculty tend to value TEP applicants' performance in college STEM courses relatively more than STEM education faculty, while STEM education faculty tend to value applicants' race and ethnicity relatively more than Arts and Sciences faculty.

Correctional education is a prevalent form of rehabilitation programming for prisoners in the United States. There is limited evidence, however, about the labor market returns to credentials received while incarcerated. Using incarceration, educational, and labor market data in Washington State, we study the labor market returns to GEDs and short-term vocational certificates earned in prison. We identify the returns to credentials by a difference-indifferences design that compares changes in earnings and employment for incarcerated persons who earn a credential to those who enroll in a program but fail to complete a GED or certificate. We estimate that GEDs increase post-incarceration earnings by about $450 per quarter and that vocational certificates increase earnings by about $250 per quarter. Degree completers have higher hourly wages, are more likely to be employed, and work more hours following release. For vocational programs, earnings increases are driven by certificates in construction and manufacturing. [The research presented presented in this report uses confidential data from the Education Research and Data Center (ERDC) located within the Washington Office of Financial Management (OFM).]

Graduates of special education teacher education programs can teach in a range of special education settings, raising the potential that their training can occur in very different settings than where they find their first jobs. We follow 263 completers of Moderate Disabilities programs in Massachusetts from their field placements to their early-career teaching positions and study the characteristics of their field placements and the degree to which these are aligned with their early-career teaching positions. We also assess the degree to which alignment is associated with early-career teacher turnover. We found that many of these teachers student-taught in an inclusive setting but were hired into a self-contained special education setting and vice versa, and teachers who experienced this misalignment were more likely to leave the workforce early in their careers. Teachers who student taught with a supervising practitioner without a special education license were also more likely to leave early. Findings suggest that teachers training to educate students with learning disabilities should student teach in a setting that is aligned with where they are likely to be hired, and with a supervising practitioner who is trained in special education.

This publication was prepared as part of the Cedefop project The shift to learning outcomes: rhetoric or reality. The purpose of this research is to analyse the conceptual, structural and political factors influencing the transformation of intended learning outcomes into achieved learning outcomes. It is considered as a first step in a long-term research strategy aiming to understand better the conditions for high quality vocational education, training and learning. The research focuses on initial vocational education and training, in schools and apprenticeships, in the 27 Member States of the EU as well as Iceland and Norway. The research is divided into five separate but interlinked themes: (1) Addressing the influence of learning outcomes on pedagogical theory and tools; (2) Focusing on the influence of learning outcomes-based curricula on teaching practices (in school-based programmes); (3) Examining the influence of learning outcomes-based curricula in company training (part of apprenticeship programmes that takes place in companies); (4) Mapping and analysing the influence of learning outcomes on assessment; and (5) Developing suggestions for the way forward supporting stakeholders and policy-makers in addressing future challenges and opportunities in this area. This report examines how the learning outcomes approach is embedded in and promoted by theories of teaching and learning (epistemology, didactics, pedagogy). It aims to analyse selected theories and the way these are presented to teacher training institutions in selected regions/countries. This allows for a better understanding of the explicit and implicit assumptions made regarding the role and relevance of learning outcomes. Results of the research illustrate differences across countries in whether and how the learning outcomes approach is embedded in theories underpinning VET teacher training programmes.

Average-wage-by-major statistics have become widely available to students interested in the economic ramifications of their college major choice. However, earning a major with higher average wages does not necessarily lead individual students to higher-paying careers. This essay combines literature review with novel analysis of longitudinal student outcomes to discuss how students use average-wage-by-major statistics and document seven reasons that they may differ, sharply in some cases, from the causal wage effects of major choice. I focus on the ramifications of two-sided non-random selection into college majors, mismeasurement of longitudinal student outcomes, and failures of extrapolation between available statistics and student interests. While large differences in average wages by major are likely to indicate causal ordinal differences between fields, small differences are probably best ignored even by students with strong interest in the economic consequences of their major choices. This essay is adapted from Chapter 6 of "Metrics that Matter: Counting What's Really Important to College Students."

In this position paper, the authors lay out the imperative for equitable artificial intelligence (AI), highlighting the essential role of access-oriented institutions and calling on technology companies (both large and small), foundations, and local, state, and federal regulators to consult with the newly convened Complete College America Council on Equitable AI in Higher Education. Their belief is that equitable AI spans far beyond the risk of mis-trained data. How schools adopt or reject these tools, the priorities of AI vendors, access to resources that enable the use of these tools, and the systemic integration of historically underrepresented and underserved voices will shape whether technology amplifies privilege or fosters inclusivity. A three-fold framework is presented for understanding Equity in AI, considering not just the quality and unbiased nature of the data used to train generative AI machines but also who has access to conversations around policy and product, as well as which institutions have access to the resources and safety nets that enable innovation and experimentation in the field of AI. A disruptive new advisory council is proposed, the Complete College America Council on Equitable AI in Higher Education, composed of representatives from historically excluded institutions and, by extension, students. The authors urge policymakers, technologists, and funders to proactively consult the Council and disrupt systemic inequities by integrating AI into higher education rather than continue to perpetuate them. [This paper was created in partnership with T3 Advisory.]

This paper presents Papilusion, an AI-generated scientific text detector developed within the DAGPap24 shared task on detecting automatically generated scientific papers. We propose an ensemble-based approach and conduct ablation studies to analyze the effect of the detector configurations on the performance. Papilusion is ranked 6th on the leaderboard, and we improve our performance after the competition ended, achieving 99.46 (+9.63) of the F1-score on the official test set., Comment: to appear in "The 4th Workshop on Scholarly Document Processing @ ACL 2024" proceedings

This paper presents an overview of a survey of jet substructure observables used to study modifications of jets induced by interaction with a Quark Gluon Plasma. We further outline ideas that were presented and discussed at the \textit{New jet quenching tools to explore equilibrium and non-equilibrium dynamics in heavy-ion collisions} workshop, which was held in February 2024 at the ECT$^{*}$ in Trento, Italy. The goal of this white paper is to provide a brief report on the study of jet quenching observables earlier conducted and to present new ideas that could be relevant for future explorations., Comment: 12 pages, 2 figures, NA3:Jet-QGP group white paper

The Atacama Large Millimetre/submillimetre Array (ALMA) receivers and technical papers are cited fewer than once in every six publications. This citation shortage is impeding the development of future (sub)millimetre instruments. In an effort to facilitate the correct citations of ALMA receivers and technical papers, this memo provides a comprehensive list of papers for the scientific community. This list was produced in discussion with the scientific and instrumentalist community, based on a June 2024 survey at the European Southern Observatory workshop on the ALMA Wideband Sensitivity Upgrade, as well as with the ALMA technical staff. The authors now encourage the community to enhance their already-excellent ALMA science with the appropriate references to ensure future (sub)millimetre instrumentation can keep addressing the key questions about our Universe., Comment: 8 pages; 1 figure; 3 tables; to be updated as more papers become available. Suggestions are warmly welcomed. ALMA memos solely contains the opinion of the authors in discussion with the astronomer and instrument builder community, and does not reflect the official policy of the ALMA telescope

Due to the rapid growth of scientific publications, identifying all related reference articles in the literature has become increasingly challenging yet highly demanding. Existing methods primarily assess candidate publications from a static perspective, focusing on the content of articles and their structural information, such as citation relationships. There is a lack of research regarding how to account for the evolving impact among papers on their embeddings. Toward this goal, this paper introduces a temporal dimension to paper recommendation strategies. The core idea is to continuously update a paper's embedding when new citation relationships appear, enhancing its relevance for future recommendations. Whenever a citation relationship is added to the literature upon the publication of a paper, the embeddings of the two related papers are updated through a Temporal Graph Neural Network (TGN). A learnable memory update module based on a Recurrent Neural Network (RNN) is utilized to study the evolution of the embedding of a paper in order to predict its reference impact in a future timestamp. Such a TGN-based model learns a pattern of how people's views of the paper may evolve, aiming to guide paper recommendations more precisely. Extensive experiments on an open citation network dataset, including 313,278 articles from https://paperswithcode.com/about PaperWithCode, have demonstrated the effectiveness of the proposed approach., Comment: 10 pages, 4 figures, accepted by SDU@AAAI-2024. The AAAI Workshop on Scientific Document Understanding (2024)

We conducted an experiment during the review process of the 2023 International Conference on Machine Learning (ICML) that requested authors with multiple submissions to rank their own papers based on perceived quality. We received 1,342 rankings, each from a distinct author, pertaining to 2,592 submissions. In this paper, we present an empirical analysis of how author-provided rankings could be leveraged to improve peer review processes at machine learning conferences. We focus on the Isotonic Mechanism, which calibrates raw review scores using author-provided rankings. Our analysis demonstrates that the ranking-calibrated scores outperform raw scores in estimating the ground truth ``expected review scores'' in both squared and absolute error metrics. Moreover, we propose several cautious, low-risk approaches to using the Isotonic Mechanism and author-provided rankings in peer review processes, including assisting senior area chairs' oversight of area chairs' recommendations, supporting the selection of paper awards, and guiding the recruitment of emergency reviewers. We conclude the paper by addressing the study's limitations and proposing future research directions., Comment: See more details about the experiment at https://openrank.cc/

This paper presents CodeRefine, a novel framework for automatically transforming research paper methodologies into functional code using Large Language Models (LLMs). Our multi-step approach first extracts and summarizes key text chunks from papers, analyzes their code relevance, and creates a knowledge graph using a predefined ontology. Code is then generated from this structured representation and enhanced through a proposed retrospective retrieval-augmented generation approach. CodeRefine addresses the challenge of bridging theoretical research and practical implementation, offering a more accurate alternative to LLM zero-shot prompting. Evaluations on diverse scientific papers demonstrate CodeRefine's ability to improve code implementation from the paper, potentially accelerating the adoption of cutting-edge algorithms in real-world applications.

Machine learning is widely utilized across various industries. Identifying the appropriate machine learning models and datasets for specific tasks is crucial for the effective industrial application of machine learning. However, this requires expertise in both machine learning and the relevant domain, leading to a high learning cost. Therefore, research focused on extracting combinations of tasks, machine learning models, and datasets from academic papers is critically important, as it can facilitate the automatic recommendation of suitable methods. Conventional information extraction methods from academic papers have been limited to identifying machine learning models and other entities as named entities. To address this issue, this study proposes a methodology extracting tasks, machine learning methods, and dataset names from scientific papers and analyzing the relationships between these information by using LLM, embedding model, and network clustering. The proposed method's expression extraction performance, when using Llama3, achieves an F-score exceeding 0.8 across various categories, confirming its practical utility. Benchmarking results on financial domain papers have demonstrated the effectiveness of this method, providing insights into the use of the latest datasets, including those related to ESG (Environmental, Social, and Governance) data., Comment: 10 pages, 8 figures

We describe the design, construction and characterisation of paper based devices to perform spirometry, a standard test for lung function assessment. In this device the instantaneous flowrate of the incoming breath from a person gets transformed to a specific acoustic frequency. In this manner, the time course of the persons breath profile is mapped to a time varying acoustic signal. The captured acoustic signal can be converted to standard spirometry curves and relevant parameters can be extracted as we describe in this paper. We compared our device with commercially available devices and showed that these paper based devices provide similar performance. These devices have advantage of low cost and simplicity of operation compared to currently available commercial products.

Teaching Computer Science (CS) by having students write programs by hand on paper has key pedagogical advantages: It allows focused learning and requires careful thinking compared to the use of Integrated Development Environments (IDEs) with intelligent support tools or "just trying things out". The familiar environment of pens and paper also lessens the cognitive load of students with no prior experience with computers, for whom the mere basic usage of computers can be intimidating. Finally, this teaching approach opens learning opportunities to students with limited access to computers. However, a key obstacle is the current lack of teaching methods and support software for working with and running handwritten programs. Optical character recognition (OCR) of handwritten code is challenging: Minor OCR errors, perhaps due to varied handwriting styles, easily make code not run, and recognizing indentation is crucial for languages like Python but is difficult to do due to inconsistent horizontal spacing in handwriting. Our approach integrates two innovative methods. The first combines OCR with an indentation recognition module and a language model designed for post-OCR error correction without introducing hallucinations. This method, to our knowledge, surpasses all existing systems in handwritten code recognition. It reduces error from 30\% in the state of the art to 5\% with minimal hallucination of logical fixes to student programs. The second method leverages a multimodal language model to recognize handwritten programs in an end-to-end fashion. We hope this contribution can stimulate further pedagogical research and contribute to the goal of making CS education universally accessible. We release a dataset of handwritten programs and code to support future research at https://github.com/mdoumbouya/codeocr

I analyze the evolution of papers certified by the Graphics Replicability Stamp Initiative (GRSI) to be reproducible, with a specific focus on the subset of publications that address visualization-related topics. With this analysis I show that, while the number of papers is increasing overall and within the visualization field, we still have to improve quite a bit to escape the replication crisis. I base my analysis on the data published by the GRSI as well as publication data for the different venues in visualization and lists of journal papers that have been presented at visualization-focused conferences. I also analyze the differences between the involved journals as well as the percentage of reproducible papers in the different presentation venues. Furthermore, I look at the authors of the publications and, in particular, their affiliation countries to see where most reproducible papers come from. Finally, I discuss potential reasons for the low reproducibility numbers and suggest possible ways to overcome these obstacles. This paper is reproducible itself, with source code and data available from github.com/tobiasisenberg/Visualization-Reproducibility as well as a free paper copy and all supplemental materials at osf.io/mvnbj., Comment: 9 pages plus appendix; 12 figures plus 14 figures in the appendix

Counterfeit products cause severe harm to public safety and health by penetrating untrusted supply chains. Numerous anti-counterfeiting techniques have been proposed, among which the use of inherent, unclonable irregularities of paper surfaces has shown considerable potential as a high-performance economical solution. Prior works do not consider supply chains cohesively, either focusing on creating or improving unclonable identifiers or on securing digital records of products. This work aims to systematically unify these two separate but connected research areas by comprehensively analyzing the needs of supply chains. We construct a generalized paper-based authentication framework and identify important shortcomings and promising ideas in the existing literature. Next, we do a stage-wise security analysis of our consolidated framework by drawing inspiration from works in signal processing, cryptography, and biometric systems. Finally, we examine key representative scenarios that illustrate the range of practical and technical challenges in real-world supply chains, and we outline the best practices to guide future research.

The impact of research papers, typically measured in terms of citation counts, depends on several factors, including the reputation of the authors, journals, and institutions, in addition to the quality of the scientific work. In this paper, we present an approach that combines natural language processing and machine learning to predict the impact of papers in a specific journal. Our focus is on the text, which should correlate with impact and the topics covered in the research. We employed a dataset of over 40,000 articles from ACS Applied Materials and Interfaces spanning from 2012 to 2022. The data was processed using various text embedding techniques and classified with supervised machine learning algorithms. Papers were categorized into the top 20% most cited within the journal, using both yearly and cumulative citation counts as metrics. Our analysis reveals that the method employing generative pre-trained transformers (GPT) was the most efficient for embedding, while the random forest algorithm exhibited the best predictive power among the machine learning algorithms. An optimized accuracy of 80\% in predicting whether a paper was among the top 20% most cited was achieved for the cumulative citation count when abstracts were processed. This accuracy is noteworthy, considering that author, institution, and early citation pattern information were not taken into account. The accuracy increased only slightly when the full texts of the papers were processed. Also significant is the finding that a simpler embedding technique, term frequency-inverse document frequency (TFIDF), yielded performance close to that of GPT. Since TFIDF captures the topics of the paper we infer that, apart from considering author and institution biases, citation counts for the considered journal may be predicted by identifying topics and "reading" the abstract of a paper.

Most published Information Systems research are of the behavioral science research (BSR) category rather than the design science research (DSR) category. This is due in part to the BSR orientation of many IS doctoral programs, which often do not involve much technical courses. This includes IS doctoral programs that train Information and Communication Technologies for Development (ICT4D) researchers. Without such technical knowledge many doctoral and postdoctoral researchers will not feel confident in engaging in DSR research. Given the importance of designing artifacts that are appropriate for a given context, an important question is how can ICT4D and other IS researchers increase their IS technical content knowledge and intimacy with the DSR process. In this paper we present, a process for reviewing DSR papers that has as its objectives: enhancing technical content knowledge, increasing knowledge and understanding of approaches to designing and evaluating IS/IT artifacts, and facilitating the identification of new DSR opportunities. This process has been applied for more than a decade at a USA research university., Comment: 44 pages, 9 Figures

Identifying significant references within the complex interrelations of a citation knowledge graph is challenging, which encompasses connections through citations, authorship, keywords, and other relational attributes. The Paper Source Tracing (PST) task seeks to automate the identification of pivotal references for given scholarly articles utilizing advanced data mining techniques. In the KDD CUP OAG-Challenge PST track, we design a recommendation-based framework tailored for the PST task. This framework employs the Neural Collaborative Filtering (NCF) model to generate final predictions. To process the textual attributes of the papers and extract input features for the model, we utilize SciBERT, a pre-trained language model. According to the experimental results, our method achieved a score of 0.37814 on the Mean Average Precision (MAP) metric, outperforming baseline models and ranking 11th among all participating teams. The source code is publicly available at https://github.com/MyLove-XAB/KDDCupFinal., Comment: KDD CUP 2024 OAG-Challenges, Paper Source Tracing, Technical Report of Team AoboSama @ KDD CUP 2024. August 25--29, 2024. Barcelona, Spain