Search

Prediction algorithms are used across public policy domains to aid in the identification of at-risk individuals and guide service provision or resource allocation. While growing research has investigated concerns of algorithmic bias, much less research has compared algorithmically-driven targeting to the counterfactual: human prediction. We compare algorithmic and human predictions in the context of a national college advising program, focusing in particular on predicting high-achieving, lower-income students' college enrollment quality. College advisors slightly outperform a prediction algorithm; however, greater advisor accuracy is concentrated among students with whom advisors had more interactions. The algorithm achieved similar accuracy among students lower in the distribution of interactions, despite advisors having substantially more information. We find no evidence that the advisors or algorithm exhibit bias against vulnerable populations. Our results suggest that, especially at scale, algorithms have the potential to provide efficient, accurate, and unbiased predictions to target scarce social services and resources.

Nearly half of students who enter college do not graduate. The majority of efforts to increase college completion have focused on supporting students before or soon after they enter college, yet many students drop out after making significant progress towards their degree. In this paper, we report results from a multi-year, large-scale experimental intervention conducted across five states and 20 broad-access, public colleges and universities to support students who are late in their college career but still at risk of not graduating. The intervention provided these "near-completer" students with personalized text messages that encouraged them to connect with campus-based academic and financial resources, reminded them of upcoming and important deadlines, and invited them to engage (via text) with campus-based advisors. We find little evidence that the message campaign affected academic performance or attainment in either the full sample or within individual higher education systems or student subgroups. The findings suggest low-cost nudge interventions may be insufficient for addressing barriers to completion among students who have made considerable academic progress.

Nearly half of students who enter college do not graduate. The majority of efforts to increase college completion have focused on supporting students before or soon after they enter college, yet many students drop out after making significant progress towards their degree. In this paper, we report results from a multi-year, large-scale experimental intervention conducted across five states and 20 broad-access, public colleges and universities to support students who are late in their college career but still at risk of not graduating. The intervention provided these "near-completer" students with personalized text messages that encouraged them to connect with campus-based academic and financial resources, reminded them of upcoming and important deadlines, and invited them to engage (via text) with campus-based advisors. We find little evidence that the message campaign affected academic performance or attainment in either the full sample or within individual higher education systems or student subgroups. The findings suggest low-cost nudge interventions may be insufficient for addressing barriers to completion among students who have made considerable academic progress.

We combine a large multi-site randomized control trial with administrative and survey data to demonstrate that intensive advising during high school and college leads to large increases in bachelor's degree attainment. Novel causal forest methods suggest that these increases are driven primarily by improvements in the quality of initial enrollment. Program effects are consistent across sites, cohorts, advisors, and student characteristics, suggesting the model is scalable. While current and proposed investments in postsecondary education focus on cutting costs, our results suggest that investment in advising is likely to be a more efficient route to promote bachelor's degree attainment.

Growing experimental evidence demonstrates that low-touch informational, nudge, and virtual advising interventions are ineffective at improving postsecondary educational outcomes for economically-disadvantaged students at scale. Intensive in-person college advising programs are a considerably higher-touch and more resource intensive strategy; some programs provide students with dozen of hours of individualized assistance starting in high school and continuing through college, and can cost thousands of dollars per student served. Despite the magnitude of this investment, causal evidence on these programs' impact is quite limited, particularly for programs that serve Hispanic students, the fastest growing segment of U.S. college enrollees. We contribute new evidence on the impact of intensive college advising programs through a multi-cohort RCT of College Forward, which provides individualized advising from junior year of high school through college for a majority Hispanic student population in Texas. College Forward leads to a 7.5 percentage point increase in enrollment in college, driven entirely by increased enrollment at four-year universities. Students who receive College Forward advising are nearly 12 percentage points more likely to persist to their third year of college. While more costly and harder to scale than low-touch interventions, back of the envelope calculations suggest that the benefit from increased college graduation likely induced by the program outweighs operating costs in less than two years following college completion.

The COVID-19 pandemic led to an abrupt shift from in-person to virtual instruction in Spring 2020. Using a difference-in-differences framework that leverages within-course variation on whether students started their Spring 2020 courses in person or online, we estimate the impact of this shift on the academic performance of Virginia's community college students. We find that the shift to virtual instruction resulted in a 6.7 percentage point decrease in course completion, driven by increases in both course withdrawal and failure. Faculty experience teaching a course online did not mitigate the negative effects of moving to virtual instruction.

Student loan borrowing for higher education has emerged as a top policy concern. Policy makers at the institutional, state, and federal levels have pursued a variety of strategies to inform students about loan origination processes and how much a student has cumulatively borrowed, and to provide students with greater access to loan counseling. We conducted an experiment to evaluate the impact of an outreach campaign that prompted loan applicants at a large community college to make informed and active borrowing decisions and that offered them access to remote, one-on-one assistance from a loan counselor. The intervention led students to reduce their unsubsidized loan borrowing by 7 percent, resulted in worse academic performance, and increased the likelihood of loan default during the three years after the intervention occurred. Our results suggest policy makers and higher education leaders should carefully examine the potential unintended consequences of efforts to reduce student borrowing, particularly in light of growing evidence regarding the counter-intuitive positive relationship between reduced borrowing levels and worse student academic and financial outcomes. [Additional support for this report was provided by ideas42.]

For most students from low- or moderate-income families, successfully completing the Free Application for Federal Student Aid (FAFSA) is a crucial gateway on the path to college access. However, FAFSA filing and income verification tasks pose substantial barriers to college access for low-income students. In this paper, the authors report on a pair of interventions that utilize automated, text-based outreach to: (1) provide students and families with customized information about the importance of and their status on completing the FAFSA; (2) simplify information for students and families about how to complete the FAFSA; and (3) connect students and families to personalized counseling assistance to complete the FAFSA as well as the subsequent verification process, if selected. Data from the study points to the benefit of text-based outreach as a low-cost and readily scalable strategy for improving student completion of important college-going milestones, such as timely FAFSA filing. Tables and figures are appended.

Lack of information and advising prior to college matriculation may contribute to poor post-secondary outcomes among non-traditional students. We conducted a large-scale, multi-arm field experiment with the U.S. Army to investigate whether a package of research-based personalized information and access to advising affects postsecondary choices and attainment among a large non-traditional adult population. We find no impact of the intervention on whether veterans enroll in college, on the quality of their college enrollment, or on their persistence in college. Our results suggest that influencing non-traditional populations' educational decisions and outcomes will require substantially more intensive programs.

While college enrollment among low-income students has increased steadily over the last decade, the share of students from the lowest-income families that enroll in college continues to lag considerably behind college entry rates among the highest income students. Furthermore, gaps in college completion by family income have only widened over time. Historically, policy interventions to ameliorate socioeconomic inequalities in college entry and success have focused primarily on increasing college access among students from economically-disadvantaged backgrounds. A more recent set of policy interventions has emerged to: (1) guide students to choose colleges where they have a good probability of earning a degree without incurring excessive debt; and (2) provide ongoing support to students once they have matriculated in college. Though community-based college advising programs have existed for decades, there is relatively little causal evidence documenting their impact on important student outcomes, including the quality and affordability of institution at which students enroll and whether they persist and succeed in college. This study evaluates the impact of an intensive college advising program on low-income students' college enrollment and persistence. The analyses was conducted with Bottom Line, a non-profit organization that provides intensive college advising to students during their senior year of high school and works with students to develop lists of well-matched colleges and universities to which they can apply. The sample consisted of nearly 3,000 Bottom Line applicants from the classes of 2010 through 2012 who had valid (i.e. non-missing) GPAs and were successfully merged to the Massachusetts Department of Elementary and Secondary Education's data. Data for this analysis came from Bottom Line, from the Massachusetts Department of Elementary and Secondary Education (DESE), and from the Integrated Postsecondary Education Data System (IPEDS). Findings include: (1) Intensive college advising offered by Bottom Line induces a substantial number of students to enroll in one of the encouraged four-year colleges rather than in discouraged four-year colleges or two-year colleges; (2) This effect is particularly strong for students from families where English is not the first language; (3) Treatment reduces the average net price of the institutions students are attending; and (4) There is suggestive but not conclusive evidence of increases in overall four-year college enrollment and persistence through the first two years of college. The authors argue that this evidence suggests that intensive college advising can have meaningful impacts on college enrollment decisions, and may improve persistence and degree completion.

In this article, the authors address what is called the "summer melt," where academically-accomplished, college-intending high school graduates fail to enroll anywhere in the year following high school, or do not attend the quality of institution (measured, for instance, by graduation rates) at which they have the academic credentials to be admitted. The authors' research focused on investigating whether college-ready, college-intending recent high school graduates benefit when high schools and universities collaborate to support their transition to college. Some questions included: (1) Do students who receive proactive outreach from a high school- or college-based counselor during the summer enroll in college at higher rates than students who do not receive outreach?; (2) Are students differentially responsive to outreach from counselors stationed at a university versus counselors stationed at their high school? ; and (3) Are student groups less represented on the University of New Mexico campus more responsive to proactive outreach than students that are more represented on campus? The study demonstrated that student groups traditionally underrepresented in higher education, such as Latino males, are more susceptible to having their college plans fall apart during the summer after high school graduation. In addition, the study also showed that concentrated and targeted outreach to underrepresented groups during the summer months can have a profound effect on whether they successfully matriculate, increasing their enrollment by 13 percent. Outreach from counselors stationed at the college side was particularly effective, suggesting that proactive communication and the offer of support from students' intended college may helpfully reinforce students' sense of belonging at and welcome from higher education institutions. The following are appended: (1) References; and (2) Tables and Figures.

The fastest growing supply of jobs in the United States today is in Science, Technology, Engineering, and Mathematics (STEM) fields. Yet despite the availability of work in STEM, there is not a sufficient supply of workers to fill open positions. Amidst the growing demand for STEM workers, educational achievement and attainment in STEM fields in the United States are moving in the opposite direction. The achievement of secondary school students in mathematics and science is declining, with less than one-third of U.S. eighth graders now proficient in these subjects; there is a concurrent decline in interest in STEM fields among many middle and high school students (PCAST, 2010). Equally troubling, 60 percent of all college students and more than 75 percent of Black and Latino college students who indicate initial interest in pursuing STEM fields do not persist to the point of earning a STEM degree (PCAST, 2010). An important policy question to consider is what role, if any, financial barriers play in dissuading students from pursuing or completing college study in STEM disciplines. In this paper, the authors offer new evidence of the impact of need-based financial aid on whether students pursue and complete courses and degrees in STEM fields. Their analysis focuses on Florida high school graduates in the 1999-2000 academic year who submitted a Free Application for Federal Student Aid (FAFSA) application, whose Expected Family Contribution (EFC) falls between $590 and $2,590, and who demonstrate readiness for postsecondary study in STEM disciplines. Four tables are appended.

Gaps in average college success among students of differing backgrounds have persisted in the United States for decades. One of the primary ways governments have attempted to ameliorate such gaps is by providing need-based grants, but little evidence exists on the impacts of such aid on longer term outcomes such as college persistence and degree completion. We examine the effects of the Florida Student Access Grant (FSAG) using a regression-discontinuity strategy and exploiting the cutoff used to determine eligibility. Grant eligibility had a positive effect on college attendance at public four-year institutions. Eligibility for the FSAG also increased early persistence and the cumulative number of college-level credits students earned in their first four years. Most importantly, FSAG receipt increased the likelihood of bachelor's degree receipt within six years at a public college or university by 4.6 percentage points--a 22 percent increase among students near the eligibility cutoff. The results are robust to sensitivity analyses. An appendix contains 1 figure and 3 tables.

While considerable effort has been invested to increase FAFSA completion among high school seniors, there has been much less investment to ensure that college freshmen re-apply for financial aid. Text messaging is a promising approach to inform students of important stages in the financial aid re-application process and to connect them to professional assistance when they need help. Building on a previous experiment, this paper examines and investigates the impact of financial aid-related prompts on whether college freshman re-apply for aid and persist into sophomore year. This study provides rigorous, experimental evidence on the impact of freshman-year financial aid text reminders on students' college persistence. The experimental sample includes students who were assigned to the summer 2012 text messaging treatment group and who were enrolled in college in Fall 2012. 774 students met these conditions. Researchers randomly assigned 387 students to receive the freshman year financial aid text messaging intervention and 384 students to the control group. By virtue of using a randomized control trial design to select which students would receive the financial aid texts, researchers are able to employ straightforward regression analyses to isolate the causal effect of the messaging campaign on students' college attainment. Preliminary data indicates a substantial response from students to the financial aid nudges. These findings are relevant both to ongoing policy efforts to increase college success and completion among low-income students and, more generally, to efforts to harness technology to improve students' educational outcomes. Figures are appended.

Despite decades of policy intervention to increase college entry among low-income students, considerable gaps by socioeconomic status remain. To date, policy makers have largely overlooked the summer after high school as an important time period in students' transition to college. In Summer 2012, the authors designed a randomized trial to investigate the efficacy of text messaging as a strategy to mitigate summer attrition. In collaboration with five urban school districts, implemented was a text messaging campaign in which the authors sent students (and in four school districts, their parents) a series of 8-10 text message reminders of key tasks to complete over the summer. The reminders were customized to the institutions at which students intended to enroll and provided recipients with the option of requesting help from a counselor. Approximately 25 percent of treatment group students clicked through at least one of the customized links included in the messages, while 35-50 percent of recipients responded to at least one message. Of these respondents, a considerable share requested a counselor meeting. In addition, counselors commonly indicated that the text messaging campaign allowed them to focus more time on providing students with college-related services rather than doing outreach. These findings are relevant both to ongoing policy efforts to increase college-going among low-income students, and more generally to efforts to harness technology to improve students' educational outcomes.

Research has shown that low-income students especially struggle with evaluating financial aid offers and completing all necessary requirements to enroll after paying a deposit to a particular college in the spring (Authors, 2009). Motivated by these empirical results, the authors conducted three studies that showed students who were explicitly offered summer assistance from their high school counselors increased the probability of fall college enrollment, suggesting that summer support may be a cost-effective intervention for promoting college enrollment among low-income students. However, several questions remain regarding how summer outreach and counseling could be conducted most efficiently and effectively. The focus of this paper is on investigating the impact of summer support provided by college-aged peer mentors on timely college matriculation among recent high school graduates with articulated college plans. Outreach from peer mentors is a more cost-effective strategy than outreach from professional counselors, and the authors hypothesize that students may be particularly likely to respond to college peers to whom they may feel they can better relate. Through this intervention, peer mentors provided outreach and support to students in four distinct locations: the public school districts in Boston, Lawrence and Springfield (Massachusetts) and a network of charter schools in the Philadelphia metropolitan area. Across the sites, 20 peer mentors provided proactive outreach and the offer of support to 907 students randomly selected from a totally sample of 2,209 students. Peer mentors were each allocated caseloads of students identified as college-intending, based on high school exit surveys and other information-gathering tools utilized by the partnering organizations. The summer counseling interventions to date have provided evidence that outreach and the offer of support in the months after high school graduation positively impact timely college going among college-intending high school graduates. A fuller description of conclusions and recommendations associated with peer mentors as an outreach model relies on postsecondary enrollment data which the authors anticipate receiving by December.