Search

Multiple measures assessment (MMA) has gained considerable momentum over the past decade as an alternative to traditional test-based procedures for placing incoming students into developmental or college-level coursework in math and English at broad-access colleges. Compared to standardized tests, which measure student performance at a single point in time, MMA (which often emphasizes high school GPA as a measure) provides a more holistic picture of students' academic preparation. Despite positive impacts on student outcomes that have been found by recent research on MMA, questions remain about whether the positive effects of MMA are sustained over time. This study--a follow-up to prior research using the same sample of students--employs a randomized controlled trial to investigate whether algorithmic MMA placement used at seven State University of New York (SUNY) community colleges led to better student outcomes, for up to four and a half years after randomization, than a system based on test scores alone. Nearly 13,000 incoming students who arrived at the seven colleges in fall 2016, spring 2017, and fall 2017 took placement tests and were randomly assigned to be placed using either the status quo method (business-as-usual group) or the alternative, algorithmic MMA method (program group). Using this sample, we estimate the overall treatment effects on placement into, enrollment in, and completion of college-level math and English as well as effects on other outcomes. We conduct similar analyses on race/ethnicity, Pell recipient status, and gender subgroups. We also descriptively examine the proportion of program group students who were bumped up (i.e., their placement changed from a developmental course placement to a college-level course placement) and bumped down (i.e., their placement changed from a college-level course placement to a developmental course placement) by the MMA algorithm, and we perform a cost-effectiveness analysis. We find that the MMA method used at the colleges improved access to and success in college-level courses and that lower cut scores in English rather than math are associated with larger and longer lasting impacts on completion of college-level coursework. While MMA improved outcomes among student subgroups, it had little to no impact on gaps in outcomes between subgroups. We also find that bumped-up students had substantially better outcomes in both math and English, while bumped-down students had substantially worse outcomes. Our results suggest that increased access to college-level courses is the driving factor in the positive outcomes experienced by program group students and that placement into standalone developmental courses can have detrimental effects on student outcomes. In the discussion of the study's results, we make recommendations for adopting MMA at colleges. Implemented together with other initiatives to support students, MMA can be a first step on the path to success for incoming students. [For the accompanying research brief, see ED632528.]

This appendix is a supplement to the report "Possible Ways of Increasing College Access Among Adults from Underserved Backgrounds: A Study of College Transition Text-Based Messaging." It provides additional information about the study of Text Ed--the messaging strategy evaluated for its ability to increase college enrollment and Free Application for Federal Student Aid completion rates among clients of Education Opportunity Centers. This includes details about the Text Ed messages, how the study was designed and conducted, detailed impact and implementation analyses, and exploratory analyses to further elucidate the findings and suggest avenues for future study. The content of the appendix is referenced as applicable within the main report. This document presents the following four appendixes that accompany the study: (1) Additional Details on Text Ed; (2) Additional Information About How the Study was Designed and Implemented; (3) Supplemental Tables for the Impact Analysis; and (4) Detailed Implementation Analysis. [For the full Evaluation Report, see ED628025. For the Study Highlights, see ED628026.]

Applying to college and securing financial aid are complex processes that can be barriers to college enrollment, particularly for individuals not entering directly after high school. These adults can seek out support from various organizations, including the U.S. Department of Education-funded Educational Opportunity Centers (EOCs), which operate from offices located within select colleges and community-based organizations. The EOCs primarily target their help to adults with low incomes and those who are potential first-generation college-goers. This study examines whether adding low-cost, text-based messaging can help EOCs more effectively work with their clients to increase clients' completion of the Free Application for Federal Student Aid (FAFSA) and college enrollment. About 3,500 EOC college-intending clients from 18 EOCs across the country were randomly divided into two groups: one received regular EOC services, and the other received EOC services plus a set of personalized, pre-programmed text messages focused on how to secure financial aid, complete key college enrollment steps, and navigate other potential barriers to college entry. The study compared the two groups to determine the effectiveness of the college transition text messaging. Key findings include: (1) Adding personalized text messaging to existing EOC services did not increase clients' rates of college enrollment or FAFSA completion; and (2) Although many clients completed important college-going tasks such as applying to college and submitting the FAFSA, 4 out of every 10 clients did not enroll in college. [For the Study Highlights, see ED628026. For the appendix, see ED628028.]

In response to research raising concerns about developmental placement, new placement systems and curricular models designed to increase the number of students placed into entry-level college courses have emerged in recent years. These systems aim to more accurately identify students who may benefit from some type of developmental instruction and those who are ready for entry-level college courses. This brief presents the findings of CAPR's exploratory study of informed self-placement (ISP), a placement system in which colleges provide information about placement policies, available courses, and other relevant topics to engage students as active participants in their own placement. ISP, which is also called guided self-placement or directed self-placement, is of interest because it does not rely solely, if at all, on standardized test scores, which research shows are not reliable predictors of performance in college. Moreover, many colleges adopted ISP during the COVID-19 pandemic, when this study began, because they experienced difficulty administering standardized tests and were seeking placement methods that could be easily used in a virtual setting. The study examined course enrollment and completion trends among students placed using ISP practices and sought to document varied approaches to ISP. The researchers collected demographic and outcome data on students' math and English course enrollments and completions over the last five academic years at three Nevada colleges offering associate and bachelor's degrees. They also interviewed assessment and placement scholars and representatives of higher education systems and institutions currently implementing some form of ISP. This brief provides a taxonomy of various placement systems, shares descriptive data on course enrollment and completion, and identifies important equity and access considerations for states and institutions interested in implementing ISP. Although more research is needed to understand the causal impacts of ISP on student outcomes, the data suggest that ISP has the potential to improve students' access to college-level coursework.

Colleges throughout the United States are evaluating the effectiveness of their strategies to place entering students into college-level or developmental education courses. Developmental, or remedial, courses are designed to advance the reading, writing, and math skills of students who are deemed academically underprepared for college-level courses. Placements have traditionally been determined through standardized placement testing; however, through evaluating additional types of placement tests, high school transcripts, and evaluations of student motivation, multiple measures assessments (MMAs) are becoming an increasingly popular tool to place students with greater nuance. Colleges must decide what measures to include, and how to combine them. This study was developed to add to the understanding of the implementation, cost, and efficacy of an MMA system using locally determined rules. As part of a randomized controlled trial, the study team evaluated MMA programs and observed 17,203 student performances across five colleges in Minnesota and Wisconsin over the course of the fall 2018, spring 2019, and fall 2019 semesters. This report examines the impacts of MMA on math and English gatekeeper course completion and college-level credit accumulation three semesters after students' initial placement. This report also analyzes the predictive utility of high school GPA, placement tests, and the noncognitive assessments used by the study colleges.

Postsecondary education is widely seen as a necessity in the modern economy. Yet students at community colleges often face steep odds when it comes to completing a degree. Community colleges serve many low-income and first-generation students as well as students of color, all of whom must contend with many obstacles to success. At the same time, two-year colleges are severely underfunded, and therefore are unable to provide the level of personal support that many students require. MDRC and the Detroit Regional Chamber partnered in 2016 to create the Detroit Promise Path (DPP), an evidence-based student services program designed to help more Detroit high school graduates--among the nation's most underserved students--enroll and persist in college, accumulate credits, graduate, and potentially transfer to a four-year program. DPP builds on an existing scholarship program called the Detroit Promise, launched in 2013, which covers community college tuition and fees for up to three years of attendance. At the heart of DPP are campus coaches who help students acclimate to college, proactively reach out to them with help and reminders about tasks and deadlines, and offer a sympathetic ear to young people who may be grappling with personal challenges--all with the goal of keeping them in school and on track to graduate. This executive summary presents an overview of findings from MDRC's randomized controlled trial evaluation of DPP at five Detroit community colleges. [Additional funding for this report was provided by the Michigan Education Excellence Foundation. For the full report, see ED611769.]

Postsecondary education is widely seen as a necessity in the modern economy. Yet students at community colleges often face steep odds when it comes to completing a degree. Community colleges serve many low-income and first-generation students as well as students of color, all of whom must contend with many obstacles to success. At the same time, two-year colleges are severely underfunded, and therefore are unable to provide the level of personal support that many students require. MDRC and the Detroit Regional Chamber partnered in 2016 to create the Detroit Promise Path (DPP), an evidence-based student services program designed to help more Detroit high school graduates--among the nation's most underserved students--enroll and persist in college, accumulate credits, graduate, and potentially transfer to a four-year program. DPP builds on an existing scholarship program called the Detroit Promise, launched in 2013, which covers community college tuition and fees for up to three years of attendance. At the heart of DPP are campus coaches who help students acclimate to college, proactively reach out to them with help and reminders about tasks and deadlines, and offer a sympathetic ear to young people who may be grappling with personal challenges--all with the goal of keeping them in school and on track to graduate. This report presents findings from MDRC's randomized controlled trial evaluation of DPP at five Detroit community colleges. A student survey found that nearly 90 percent of respondents rated the program as valuable or highly valuable. The evaluation found that more students in the DPP program stayed enrolled in school and earned more credits, compared with students who were offered the Promise scholarship alone. However, at the three-year mark, there was no evidence of an impact on degrees earned. It is clear that promoting college "access" is not enough. Programs must also tackle progress--helping students stay in school and get to graduation. [Additional funding for this study was provided by the Michigan Education Excellence Foundation. For the Executive Summary, see ED611776.]

How should colleges determine whether students are placed into developmental or college-level courses? Each year, colleges place millions of students into developmental math and English courses upon enrollment. To do so, colleges most often use a high-stakes placement test, which numerous research studies have shown to be highly inaccurate in determining how well students are prepared for college. As a result, these tests "underplace" many students into developmental education classes who would have been successful immediately if they had taken college credit-bearing courses instead. Developmental education courses are designed to give students the skills they need for success in college-level courses, but they also delay students' enrollment in credit-bearing coursework, lengthen the time it takes them to earn a degree, and may decrease the chances they will ever graduate. Colleges could boost incoming students' college-level course pass rates by improving the assessment tools they use to place those students, with the goal of minimizing underplacement and increasing the number of students taking college-level courses. Using more than one measure to assess students' skills--a strategy known as a multiple measures assessment (MMA)--can be an excellent way to achieve this goal. This brief summarizes the findings from a study of the impacts of two MMA models at seven 2-year state colleges in New York and at four 2-year state colleges in Minnesota as an alternative to the high-stakes testing that colleges typically use to make placement decisions. The study found that students who are placed into college-level courses using MMA are more likely to complete gatekeeper courses--basic introductory or prerequisite college-level courses--than their counterparts who are placed into developmental courses using placement tests. These findings held for placements in math and English courses in the first semester and after three semesters. The study also found that student success rates can improve when MMA is applied and students who would have otherwise been placed into developmental education courses are instead referred to college-level courses.

Virtually all community colleges and more than 90 percent of public four-year colleges use the results of placement tests--either alone or in concert with other information--to determine whether students are ready for college-level coursework or need remedial help in math or English. Evidence suggests that placement tests do a poor job of indicating which students need remediation. The Center for the Analysis of Postsecondary Readiness (CAPR) is studying an alternative placement system that uses multiple measures--including both placement test scores and high school GPAs--in predictive algorithms to place incoming students into remedial or college-level courses. Seven community colleges in the State University of New York system participated in the random assignment study to determine whether multiple measures placement leads to better student outcomes than a system based on test scores alone. Using multiple measures placement, many more students were assigned to college-level courses. In math, gains in college-level enrollment and completion were small and short-lived. But in English, the effects were much larger and lasted through at least three semesters. Regardless of whether they were predicted to succeed, students did better when they were allowed to start in college-level courses. A report on longer-term outcomes from the study will be released in summer 2022.

The nation's community colleges play a central role in producing a more educated workforce and promoting social mobility. They serve about 40 percent of all college students and, not surprisingly, they serve a disproportionate number of low-income and underrepresented students. But most students who enter these colleges do not graduate--only about a third of entering students earn a degree or certificate within six years. Among the many programs that have attempted to increase graduation rates, one program stands out. Developed by the City University of New York (CUNY), the Accelerated Study in Associate Programs (ASAP) is a comprehensive program that provides students with up to three years of financial and academic support and other support services. Along with those services and other forms of support comes an obligation to attend full time and participate in essential program services. An experimental evaluation of CUNY ASAP found that the program nearly doubled graduation rates after three years. This report presents findings through three years from a replication of the ASAP model at three community colleges in Ohio. Low-income students were randomly assigned either to a program group, who could participate in their colleges' new programs based closely on ASAP (called the Ohio Programs), or to a control group, who could receive the usual college services. Comparing the two groups' outcomes provides an estimate of the Ohio Programs' effects. The findings from the evaluation add to a body of evidence showing that comprehensive programs that offer academic, financial, and other forms of student support for multiple years are an effective way to increase college completion rates. The fact that the earlier CUNY ASAP findings have now been replicated in Ohio suggests that ASAP in particular can serve as a national model to help students succeed.

Financial aid plays an essential role not only in allowing many students to enroll in college but also in supporting them in attaining completion and success. Often, however, the total amount of aid does not come close to covering the cost of attendance for full-time students. As a result, the majority of students enrolled at two-year public institutions report feeling financial stress related to paying for school. Students often work while attending college to cover the full cost of attendance, but time spent working can have a negative impact on their academic success. MDRC launched Aid Like A Paycheck to test whether changes to the timing of student aid disbursement could help students stretch their financial aid to cover their expenses throughout the term, and whether such a policy could improve students' academic and financial outcomes. Most colleges distribute financial aid refunds to students in one or two lump sums during the term. Aid Like A Paycheck tested an alternate approach, in which financial aid refunds were disbursed biweekly, with the goal of helping students better budget their existing financial aid. MDRC conducted a mixed-methods evaluation of incremental financial aid disbursements at two community college systems in and around Houston, Texas, and at a third system in California's rural Central Valley. At the two institutions in Texas, the study included a randomized controlled trial that gathered data from nearly 9,000 students and tracked them for up to two years. The final findings from the study indicate that biweekly disbursements do not result in substantial impacts on student outcomes: (1) Students assigned to receive biweekly disbursements and those assigned to receive their aid in the standard way received the same total amount of financial aid; (2) Biweekly disbursements reduced students' debt to the college in the first semester, but this reduction in debt was no longer evident at the end of the fourth semester; (3) On average, there is no evidence of biweekly disbursements improving students' key academic outcomes; and (4) There is little evidence that the participating colleges or the government saved money by implementing biweekly disbursements; and (5) Implementation of the policy was costlier than -- and not as simple as -- expected, even when implemented without the constraints of the randomized controlled trial design. Overall, the study suggests that incremental disbursements neither hurt students nor substantially improve their academic or financial outcomes. [To read the provisional findings of this study, "Aligning Aid with Enrollment: Interim Findings on Aid Like a Paycheck," see ED574751.]

The Aid Like A Paycheck student survey was designed to shed light on students' academic and financial attitudes, behaviors, and circumstances over the course of a semester. This supplement to MDRC's final report on Aid Like A Paycheck describes the survey, response rates, and responses to each survey item at three community college systems: two in and around Houston, Texas (San Jacinto College and the Houston Community College System), which participated in a randomized controlled trial, and one in California (the West Hills Community College District), which scaled up the intervention to implement biweekly payments for all eligible students. [For the main report, see ED595310. For "Incremental Disbursements of Student Financial Aid: Final Report on Aid Like a Paycheck. Executive Summary," see ED596622.]

Financial aid plays an essential role not only in allowing many students to enroll in college but also in supporting them in attaining completion and success. Often, however, the total amount of aid does not come close to covering the cost of attendance for full-time students. As a result, the majority of students enrolled at two-year public institutions report feeling financial stress related to paying for school. MDRC launched Aid Like A Paycheck to test whether changes to the timing of student aid disbursement could help students stretch their financial aid to cover their expenses throughout the term, and whether such a policy could improve students' academic and financial outcomes. Most colleges distribute financial aid refunds to students in one or two lump sums during the term. Aid Like A Paycheck tested an alternate approach, in which financial aid refunds were disbursed biweekly, with the goal of helping students better budget their existing financial aid. MDRC conducted a mixed-methods evaluation of incremental financial aid disbursements at two community college systems in and around Houston, Texas, and at a third system in California's rural Central Valley. At the two institutions in Texas, the study included a randomized controlled trial that gathered data from nearly 9,000 students and tracked them for up to two years. The final findings from the study indicate that biweekly disbursements do not result in substantial impacts on student outcomes. Overall, the study suggests that incremental disbursements neither hurt students nor substantially improve their academic or financial outcomes. Among the findings highlighted in this executive summary are: (1) Students assigned to receive biweekly disbursements and those assigned to receive their aid in the standard way received the same total amount of financial aid; (2) Biweekly disbursements reduced students' debt to the college in the first semester, but this reduction was no longer evident by the end of their fourth semester; (3) On average, there is no evidence of biweekly disbursements improving students' key academic outcomes; and (4) There is little evidence that the participating colleges or the government saved money by implementing biweekly disbursements. [For the main report, see ED595310. For "Incremental Disbursements of Student Financial Aid: Final Report on Aid Like a Paycheck. Supplemental Tables," see ED596623.]

While the United States has made strides in increasing college access among low-income students, college completion has remained low. Graduation rates are particularly low at the nation's community colleges, which enroll a disproportionate percentage of low-income and nontraditional college students. Seeking to address this problem, in 2014 three community colleges in Ohio -- Cincinnati State Technical and Community College, Cuyahoga Community College, and Lorain County Community College -- undertook a new strategy to help more of their lowest-performing students succeed academically. The highly successful Accelerated Study in Associate Programs (ASAP) developed by the City University of New York (CUNY) provided a model. ASAP is a comprehensive program that provides students with up to three years of financial and academic support and other support services to address multiple barriers to student success, with the goal of helping more students graduate within three years. This brief presents two-year impact, implementation, and cost findings for the pooled, full study sample in the ASAP Ohio demonstration. The findings show that students in the program group clearly outperformed the control group with respect to persistence in school, credit accumulation, and graduation. Graduation rates more than doubled: 19 percent of the program group earned a degree or credential after two years compared with 8 percent of the control group. The brief also presents some findings from analyses of the programs' implementation and costs. [This report was written with Sean Blake and Erick Alonzo.]

Evidence shows that financial aid increases college enrollment. For many students at low-cost community colleges, this aid is intended to cover more than tuition and fees; after those are paid, the remainder is paid out, or "refunded," to students to help with their living expenses while they are enrolled in school. Often, however, the total amount of aid does not come close to covering the cost of attendance for full-time students. Many still struggle to find the means to support their studies and pay their bills, and the majority of students enrolled at two-year public institutions report feeling financial stress related to paying for school. Most colleges distribute financial aid refund amounts to students in one or two lump sums during the term. "Aid Like A Paycheck" is a study of an alternative approach, in which financial aid refunds are disbursed biweekly, with the goal of helping students stretch their financial aid (including federal Pell Grants, state aid, and loans) to cover expenses throughout the term. "MDRC" is conducting a mixed-methods study of biweekly disbursements at two community colleges in the metropolitan area of Houston, Texas. The study includes qualitative research on the program's implementation and a randomized controlled trial to rigorously estimate the impacts of the policy on students' academic and financial outcomes. Results at the midpoint of the evaluation present a mixed picture: (1) The colleges were able to disburse aid biweekly as intended, but communications about the policy and about financial aid in general were often unclear to students; (2) It appears that students assigned to receive biweekly disbursements were more likely than those receiving a standard lump sum to feel that their finances caused significant stress at the start of the term. By the end of the semester, students in the two groups reported comparable levels of financial stress; (3) Biweekly disbursements reduced students' use of federal loans and debt to the college after one semester, without reducing the overall aid they received; (4) On average, there is little consistent evidence of biweekly disbursements improving students' key academic outcomes. However, at one college, students in the biweekly group experienced a 6 percentage point increase in enrollment in the second semester of school; and (5) Although the policy raised the possibility of cost savings for the colleges and the federal government, there is little evidence of such savings so far. These interim findings should be viewed with caution, but colleges interested in moving to biweekly disbursements may find useful lessons from the early implementers, as discussed in this report. In 2018 MDRC will present final results based on further research, including additional students and longer follow-up, to better understand the impacts of biweekly disbursements at the two Houston-area colleges. The final report will also include a deeper look at the program's implementation at a third community college district, where biweekly disbursements of financial aid refunds recently became the standard policy for all students. [For the full report, see ED574751.]

Evidence shows that financial aid increases college enrollment. For many students at low-cost community colleges, this aid is intended to cover more than tuition and fees; after those are paid, the remainder is paid out, or "refunded," to students to help with their living expenses while they are enrolled in school. Often, however, the total amount of aid does not come close to covering the cost of attendance for full-time students. Many still struggle to find the means to support their studies and pay their bills, and the majority of students enrolled at two-year public institutions report feeling financial stress related to paying for school. Most colleges distribute financial aid refund amounts to students in one or two lump sums during the term. "Aid Like A Paycheck" is a study of an alternative approach, in which financial aid refunds are disbursed biweekly, with the goal of helping students stretch their financial aid (including federal Pell Grants, state aid, and loans) to cover expenses throughout the term. "MDRC" is conducting a mixed-methods study of biweekly disbursements at two community colleges in the metropolitan area of Houston, Texas. The study includes qualitative research on the program's implementation and a randomized controlled trial to rigorously estimate the impacts of the policy on students' academic and financial outcomes. Results at the midpoint of the evaluation present a mixed picture: (1) The colleges were able to disburse aid biweekly as intended, but communications about the policy and about financial aid in general were often unclear to students; (2) It appears that students assigned to receive biweekly disbursements were more likely than those receiving a standard lump sum to feel that their finances caused significant stress at the start of the term. By the end of the semester, students in the two groups reported comparable levels of financial stress; (3) Biweekly disbursements reduced students' use of federal loans and debt to the college after one semester, without reducing the overall aid they received; (4) On average, there is little consistent evidence of biweekly disbursements improving students' key academic outcomes. However, at one college, students in the biweekly group experienced a 6 percentage point increase in enrollment in the second semester of school; and (5) Although the policy raised the possibility of cost savings for the colleges and the federal government, there is little evidence of such savings so far. These interim findings should be viewed with caution, but colleges interested in moving to biweekly disbursements may find useful lessons from the early implementers, as discussed in this report. In 2018 MDRC will present final results based on further research, including additional students and longer follow-up, to better understand the impacts of biweekly disbursements at the two Houston-area colleges. The final report will also include a deeper look at the program's implementation at a third community college district, where biweekly disbursements of financial aid refunds recently became the standard policy for all students. [For the executive summary, see ED574752.]

Each year, millions of Americans enroll in community colleges, seeking to develop the skills necessary to pursue a career or to transfer to a four-year institution. Community colleges serve large proportions of nontraditional, low-income, and minority students, and they are designed to provide access to a postsecondary education at a low or relatively affordable cost. Yet for most students who enter these institutions, academic success remains elusive. In 2004, Lumina Foundation and a group of partner organizations--the American Association of Community Colleges; the Community College Leadership Program at the University of Texas; the Community College Research Center (CCRC) at Columbia University; Jobs for the Future; MDC, Inc.; MDRC; and Public Agenda--launched Achieving the Dream, a bold, multiyear national initiative aimed at improving student outcomes in community colleges, particularly among low-income students and students of color. The partner organizations were selected to help Lumina design and operate Achieving the Dream, which set out to foster fundamental changes in the culture and operations of community colleges. Lumina and the founding partners sought to spur a process of institutional change through monetary and professional supports from the initiative, combined with colleges' own investments. This process centered on building a "culture of evidence"--one in which colleges routinely use evidence to help their students succeed academically. The partners theorized that undertaking broad-based institutional efforts would ultimately lead to improvements in student outcomes. Twenty-six colleges (called the "Round 1" colleges) were the first to join the initiative in 2004. In 2011, MDRC, in partnership with CCRC, published "Turning the Tide: Five Years of Achieving the Dream in Community Colleges". That report described the implementation of the initiative and trends in student outcomes among these 26 colleges from 2004 through 2009. This report, the final publication from MDRC and CCRC on the Round 1 colleges, builds on "Turning the Tide" in two ways. First, it extends the analyses of institution-wide outcomes to students who were entering the Round 1 institutions during the latter period of the colleges' five-year implementation grants, when institutions were expected to have more fully implemented many of their Achieving the Dream initiatives. Second, the report explores variation in student outcome trends at Round 1 colleges and reanalyzes the implementation data in order to inform other institutions that are undertaking reforms. This report is a retrospective study of Achieving the Dream as it was implemented between 2004 and 2009 at the first 26 colleges to participate, rather than an assessment of the initiative's direct impact on its student outcomes or current activities and programs. Indeed, the initiative now includes nearly 200 participating colleges. Overall, this report finds that average institution-wide trends in student outcomes remained relatively stable during the period of study, including during the prolonged recession that began in the United States in late 2007. Three colleges, however, stood out for gains on multiple indicators of student success. The practices of these institutions suggest possible lessons for community college practitioners, in addition to new directions for research. In particular: (1) Each college focused on specific student subgroups, and each coordinated multiple reform efforts around their chosen subgroup; (2) In later years, after gaining experience with the initial subgroups, each college expanded its new practices in order to reach larger groups of students and faculty. Targeted professional development for faculty and staff involved in the work supported this focus; and (3) One college used its reaccreditation process to help coordinate its reform efforts and to work toward establishing a common set of goals. This report concludes by discussing the lessons gleaned from the experiences of Round 1 colleges with Achieving the Dream. A section on additional analysis is appended. [This report was written with Donna Chan and Phoebe Richman. See ED516014, for "Turning the Tide: Five Years of Achieving the Dream in Community Colleges".]

Community colleges play a vital role in higher education, enrolling more than one in every three postsecondary students. While the market share of these institutions has grown over the past 50 years, students' success rates remain low. Consequently, community college stakeholders are searching with mounting urgency for strategies that increase rates of success. We evaluate the effects of one such strategy, learning communities, from a randomized trial of over 1,500 students at a large urban college in the City University of New York (CUNY) system. Findings indicate that the program's positive effects on short-term academic progress (credit accumulation) are maintained seven years after random assignment. In addition, the analysis provides some limited evidence that the program positively affected graduation rates, particularly for those students without remedial English needs, over this period. At the same time, however, there is no discernible evidence that the program improved economic outcomes. This paper concludes by offering sobering reflections on trying to detect the effects of higher education interventions on future earnings.

Empirical evidence confirms that increased education is positively associated with higher earnings across a wide spectrum of fields and student demographics (Barrow & Rouse, 2005; Card, 2001; Carneiro, Heckman, & Vytlacil, 2011; Dadgar & Weiss, 2012; Dynarski, 2008; Jacobson & Mokher, 2009; Jepsen, Troske, & Coomes, 2009; Kane & Rouse, 1995; Marcotte, 2004). College degree holders earn higher salaries and experience unemployment less frequently than those who do not have college degrees (Dadgar & Weiss, 2012), and those who attended college, regardless of degree receipt, report higher rates of job satisfaction, promotion opportunities, increased work responsibilities, and improved work performance (Hoachlander, Sikora, & Horn, 2003). For many low-income students, the pathway to higher education and increased earnings begins in community colleges, which currently enroll about one-third of American degreeseekers (Digest of Education Statistics, 2012). However, community college students' persistence and completion rates leave much room for improvement. Among students beginning at public two-year colleges, fewer than half earn a credential or transfer to a four-year institution within six years after their initial enrollment (Radford, Berkner, Wheeless, & Shepherd, 2010). Many community college students stop or drop out long before graduation day, losing out on the full complement of higher earnings and employment outcomes associated with higher education. One factor contributing to these low success rates is the fact that many students arrive on campus only to find that they are required to take non-credit-bearing courses, called "developmental" or "remedial" education courses, prior to enrolling in college-level courses (Duke & Strawn, 2008). One estimate of the prevalence of students required to complete developmental coursework suggests that among students whose first institution attended was a community college, around 60 percent took a remedial course at a postsecondary institution (Adelman, 2004; Bailey, Jeong, & Cho, 2009). Unsurprisingly, degree or certificate attainment rates among students who need developmental education are even lower than those of the general population (Attewell, Lavin, Domina, & Levey, 2006). Students in need of developmental education are not just failing to earn a credential -- the majority never complete the developmental course sequence to which they are referred (Bailey 2009). One popular strategy aimed at addressing the issues of students referred to developmental courses (as well as students in college-level courses) is called "learning communities," or LCs. The LC evaluated in this study produced positive short-term impacts on academic outcomes including credits attempted and credits earned and students' progression through the developmental reading sequence of courses. Given these positive results, similar learning communities for developmental reading and math students in community colleges were replicated and evaluated at five other schools. Tables and figures are appended.

Most community college students take in-person placement tests in English or math to determine whether they are required to take developmental (or remedial) courses, which are prerequisites for the college-level courses in those subjects. However, the COVID-19 pandemic has forced colleges to close their campuses, making in-person testing impossible. Some colleges are offering remote placement testing for students applying to enroll this summer and fall. But other colleges may not be able to surmount the technical obstacles involved in remote testing in the short term, leaving students and advisers without the test scores they would usually use to make course-placement decisions. Even before the pandemic, many colleges were using multiple pieces of information to determine course placement, rather than a single test score. Research on these multiple measures assessment placement systems offers three important insights that might help colleges navigate their new circumstances today.

This paper presents research that explores similarities and differences across six randomized controlled trials of learning communities in community colleges that were conducted by MDRC and the National Center for Postsecondary Research. Five of these studies track students' progress in the program semester and two follow-up semesters, and one study follows students for six years. These studies provide the most extensive evidence available on the promise and limitations of learning communities for improving the academic outcomes of students in community colleges. The present research examines several competing explanations for mixed findings and draws lessons to inform further research in the field. The following six community colleges were involved in the studies: (1) Community College of Baltimore County (CCBC) in suburban Maryland; (2) Hillsborough Community College in Tampa, Florida; (3) Houston Community College in Houston, Texas; (4) Kingsborough Community College in Brooklyn, New York; (5) Merced College in Merced, California; and (6) Queensborough Community College in Queens, New York. The learning communities each lasted for one semester, and consisted of four key components, although there was variation in their emphasis and implementation: (1) linked courses and student cohorts; (2) faculty collaboration; (3) instructional practices; and (4) student supports. More than 7,000 students were randomly assigned to either the program group or the control group. The researchers used several data sources: (1) the baseline information form; (2) operational site visits, field research, and instructor survey; and (3) student records. Results from the studies suggest the following: (1) One-semester learning communities can have a long-term impact and even boost graduation, as shown in the study of the Kingsborough program; and (2) The combined results of all six trials suggest that, on average, learning communities for developmental education students produce only a modest impact on credits earned in the targeted subject of English or mathematics. The research provides good tests of learning communities as they appear to be typically enacted, but not a test of the "ideal" or "advanced" learning communities described in the literature. Tables and figures are appended. [This report was written with Kelley Fong, Hannah Fresques, and Jedediah Teres.]

The Learning Communities Demonstration is a national research project that is testing the effectiveness of learning communities in six community colleges across the United States: Merced College in California; The Community College of Baltimore County (CCBC) in Baltimore, Maryland; Hillsborough Community College in Tampa, Florida; Houston Community College in Houston, Texas; Queensborough Community College in Queens, New York; and Kingsborough Community College in Brooklyn, New York. This report describes the findings from the programs at Merced and CCBC, which each offered learning communities for students in developmental English (including both developmental reading and developmental writing) with the goals articulated above in mind. Findings from the studies at the other four colleges can be found in previously published reports from the Learning Communities Demonstration (Visher, Schneider, Wathington, & Collado, 2010 details the early implementation experiences of the six colleges in the demonstration. The other impact studies from the demonstration can be found in Weiss, Visher, & Wathington, 2010; Weissman et al., 2011; Visher & Teres, 2011). Key findings from Merced and CCBC include: (1) Merced and CCBC had relatively ambitious goals for the implementation of advanced, semester-long, developmental English learning communities. In practice, a strong cohort experience was provided to students, and other aspects of the learning communities model were implemented with variation among the different links at each college. Overall, the colleges succeeded in providing the majority of program group students with an experience that was substantially different from that of their control group counterparts; (2) At Merced, learning communities students attempted and earned significantly more developmental English credits than students in the control group during the program semester. At the end of the subsequent semester, they had passed significantly more English courses than their control group counterparts; (3) At CCBC, there were no meaningful impacts on students' credit attempts or progress in developmental English; and (4) On average, neither college's learning communities program had an impact on college registration in the postprogram semester, or on cumulative credits earned. [This paper was written with Amanda Grossman. For "Learning Communities for Students in Developmental English: Impact Studies at Merced College and the Community College of Baltimore County," see ED529251.]

The Learning Communities Demonstration is a national research project that is testing the effectiveness of learning communities in six community colleges across the United States: Merced College in California; The Community College of Baltimore County (CCBC) in Baltimore, Maryland; Hillsborough Community College in Tampa, Florida; Houston Community College in Houston, Texas; Queensborough Community College in Queens, New York; and Kingsborough Community College in Brooklyn, New York. This report describes the findings from the programs at Merced and CCBC, which each offered learning communities for students in developmental English (including both developmental reading and developmental writing) with the goals articulated above in mind. Findings from the studies at the other four colleges can be found in previously published reports from the Learning Communities Demonstration (Visher, Schneider, Wathington, & Collado, 2010 details the early implementation experiences of the six colleges in the demonstration. The other impact studies from the demonstration can be found in Weiss, Visher, & Wathington, 2010; Weissman et al., 2011; Visher & Teres, 2011). Key findings from Merced and CCBC include: (1) Merced and CCBC had relatively ambitious goals for the implementation of advanced, semester-long, developmental English learning communities. In practice, a strong cohort experience was provided to students, and other aspects of the learning communities model were implemented with variation among the different links at each college. Overall, the colleges succeeded in providing the majority of program group students with an experience that was substantially different from that of their control group counterparts; (2) At Merced, learning communities students attempted and earned significantly more developmental English credits than students in the control group during the program semester. At the end of the subsequent semester, they had passed significantly more English courses than their control group counterparts; (3) At CCBC, there were no meaningful impacts on students' credit attempts or progress in developmental English; and (4) On average, neither college's learning communities program had an impact on college registration in the postprogram semester, or on cumulative credits earned. [This paper was written with Amanda Grossman. For "Learning Communities for Students in Developmental English: Impact Studies at Merced College and the Community College of Baltimore County," see ED529251.]

Improving the success of academically underprepared students who are in need of developmental (or remedial) education is a key challenge facing community colleges today. Many of these students enter college with little awareness of these institutions' expectations or a clear model for how to make effective decisions about their academic careers. To help students address these challenges, a number of colleges across the country have looked to success courses (also called study skills, student development, or new student orientation courses). This report analyzes a success course for developmental education students at Guilford Technical Community College in Greensboro, North Carolina, and its impact on students' psychosocial skills and behaviors and academic achievement. After joining Achieving the Dream: Community Colleges Count in 2004, a national organization designed to mentor colleges through an institutionwide, student success-oriented improvement process, Guilford chose to offer a revised version of its student success course to developmental education students, aimed at improving psychosocial awareness and academic achievement. Modeled on Skip Downing's "On Course" philosophy and curriculum, it placed an intensive focus on changing students' behaviors and attitudes, including increasing their awareness of their and others' emotions, understanding their own learning styles, improving time management skills, and recognizing their responsibility for their own learning. Guilford hoped that these changes in students' personal habits and behaviors might help them take better control of their academic lives, which would ultimately result in gains in achievement. This study employed random assignment methodology to examine the impact of Guilford's success course. The key findings presented in this report are: (1) Guilford's implementation of its student success course stayed true to the "On Course" philosophy, with a strong emphasis on improving students skills and habits; (2) Challenges emerged during the study in maintaining instructors' enthusiasm for teaching the course; (3) The course had a positive impact on students' self-management, interdependence, self-awareness, interest in lifelong learning, emotional intelligence, and engagement in college among students with low levels of these attributes; and (4) But the gains in efficacy did not lead to meaningful effects on students' academic achievement during the program semester or in postprogram semesters. Despite the absence of an overall effect, the program did have positive effects on the first cohort of students enrolled in the study, with students demonstrating improved grades, retention in college, and credits earned. The results of this study reveal that improvements in students' attitudes and behaviors may not necessarily translate easily into better academic outcomes, though the strength of program implementation may play an important role in these effects. Additionally, the program's limited effects suggest that community colleges should look to more comprehensive ways of improving developmental education students' academic achievement, including reforms in developmental education instruction. Appended are: (1) Technical Appendix; (2) Sensitivity Analysis; and (3) Survey Outcomes by Cohort. Individual chapters contain footnotes. (Contains 13 tables and 3 boxes.) [For "Keeping Students on Course: An Impact Study of a Student Success Course at Guilford Technical Community College. Executive Summary," see ED531181.]

Improving the success of academically underprepared students who are in need of developmental (or remedial) education is a key challenge facing community colleges today. Many of these students enter college with little awareness of these institutions' expectations or a clear model for how to make effective decisions about their academic careers. To help students address these challenges, a number of colleges across the country have looked to success courses (also called study skills, student development, or new student orientation courses). This report analyzes a success course for developmental education students at Guilford Technical Community College in Greensboro, North Carolina, and its impact on students' psychosocial skills and behaviors and academic achievement. After joining Achieving the Dream: Community Colleges Count in 2004, a national organization designed to mentor colleges through an institutionwide, student success-oriented improvement process, Guilford chose to offer a revised version of its student success course to developmental education students, aimed at improving psychosocial awareness and academic achievement. Modeled on Skip Downing's "On Course" philosophy and curriculum, it placed an intensive focus on changing students' behaviors and attitudes, including increasing their awareness of their and others' emotions, understanding their own learning styles, improving time management skills, and recognizing their responsibility for their own learning. Guilford hoped that these changes in students' personal habits and behaviors might help them take better control of their academic lives, which would ultimately result in gains in achievement. This study employed random assignment methodology to examine the impact of Guilford's success course. The key findings presented in this report are: (1) Guilford's implementation of its student success course stayed true to the "On Course" philosophy, with a strong emphasis on improving students' psychosocial skills and habits; (2) Challenges emerged during the study in maintaining instructors' enthusiasm for teaching the course; (3) The course had a positive impact on students' self-management, interdependence, self-awareness, interest in lifelong learning, emotional intelligence, and engagement in college among students with low levels of these attributes; and (4) But the gains in efficacy did not lead to meaningful effects on students' academic achievement during the program semester or in postprogram semesters. Despite the absence of an overall effect, the program did have positive effects on the first cohort of students enrolled in the study, with students demonstrating improved grades, retention in college, and credits earned. The results of this study reveal that improvements in students' attitudes and behaviors may not necessarily translate easily into better academic outcomes, though the strength of program implementation may play an important role in these effects. Additionally, the program's limited effects suggest that community colleges should look to more comprehensive ways of improving developmental education students' academic achievement, including reforms in developmental education instruction. (Contains 7 footnotes.) [For the full report, "Keeping Students on Course: An Impact Study of a Student Success Course at Guilford Technical Community College," see ED531183.]

Across the United States, community colleges offer millions of students an open-access, low-cost postsecondary education. However, of the students who enroll in community college hoping to earn a credential or transfer to a four-year institution, only about half achieve their goal within six years. For students who enter college needing developmental (remedial) education in reading, writing, or math, this rate is even lower. Learning communities, in which cohorts of students enroll in two or more linked courses together, are often employed to improve these students' success. In addition to linking courses, learning communities often incorporate other components, such as faculty collaboration, shared assignments and curricula, and connections to student support services. Merced College in California and The Community College of Baltimore County (CCBC) each developed learning communities designed to boost the academic success of their developmental English students. These colleges are two of the six in the National Center for Postsecondary Research's (NCPR) Learning Communities Demonstration, in which random assignment evaluations are being used to determine the impacts of learning communities on student success. At Merced, learning communities linked developmental English courses with a variety of other courses at the developmental and college levels. At CCBC, learning communities linked developmental English with a range of college-level courses and a weekly one-hour Master Learner session designed to support curricular integration and student learning. The key findings presented in this report are: (1) Both Merced and CCBC implemented relatively advanced learning communities. A strong cohort experience was provided to students, and other aspects of the learning communities model were implemented with variation at each college. On average, the colleges succeeded in providing program group students with an experience that was substantially different from the experience of their control group counterparts; (2) At Merced, learning communities students attempted and earned significantly more developmental English credits than students in the control group during the program semester. At the end of the subsequent semester, they had passed significantly more English courses than their control group counterparts; (3) At CCBC, there were no meaningful impacts on students' credit attempts or progress in developmental English; and (4) On average, neither college's learning communities program had an impact on college registration in the postprogram semester, or on cumulative credits earned. NCPR has now presented findings from all six colleges in the demonstration. They show that when one-semester learning communities have impacts, they tend to be concentrated in the semester in which students are enrolled in the program. The evidence to date suggests that one-semester learning communities programs by themselves are typically not sufficient to boost reenrollment or increase credit accumulation. However, this is not the final report on the demonstration; in 2012, NCPR will release a report that synthesizes the findings across all of the colleges studied and includes an additional semester of student follow-up at each college. Supplementary table and figures are appended. Individual chapters contain footnotes. (Contains 11 tables, 9 figures and 1 box.) [This paper was written with Amanda Grossman. References for the executive summary are included. For "Learning Communities for Students in Developmental English: Impact Studies at Merced College and the Community College of Baltimore County. Executive Summary," see ED529250.]

Across the United States, community colleges offer millions of students an open-access, low-cost postsecondary education. However, of the students who enroll in community college hoping to earn a credential or transfer to a four-year institution, only about half achieve their goal within six years. For students who enter college needing developmental (remedial) education in reading, writing, or math, this rate is even lower. Learning communities, in which cohorts of students enroll in two or more linked courses together, are often employed to improve these students' success. In addition to linking courses, learning communities often incorporate other components, such as faculty collaboration, shared assignments and curricula, and connections to student support services. Merced College in California and The Community College of Baltimore County (CCBC) each developed learning communities designed to boost the academic success of their developmental English students. These colleges are two of the six in the National Center for Postsecondary Research's (NCPR) Learning Communities Demonstration, in which random assignment evaluations are being used to determine the impacts of learning communities on student success. At Merced, learning communities linked developmental English courses with a variety of other courses at the developmental and college levels. At CCBC, learning communities linked developmental English with a range of college-level courses and a weekly one-hour Master Learner session designed to support curricular integration and student learning. The key findings presented in this report are: (1) Both Merced and CCBC implemented relatively advanced learning communities. A strong cohort experience was provided to students, and other aspects of the learning communities model were implemented with variation at each college. On average, the colleges succeeded in providing program group students with an experience that was substantially different from the experience of their control group counterparts; (2) At Merced, learning communities students attempted and earned significantly more developmental English credits than students in the control group during the program semester. At the end of the subsequent semester, they had passed significantly more English courses than their control group counterparts; (3) At CCBC, there were no meaningful impacts on students' credit attempts or progress in developmental English; and (4) On average, neither college's learning communities program had an impact on college registration in the postprogram semester, or on cumulative credits earned. NCPR has now presented findings from all six colleges in the demonstration. They show that when one-semester leaning communities have impacts, they tend to be concentrated in the semester in which students are enrolled in the program. The evidence to date suggests that one-semester learning communities programs by themselves are typically not sufficient to boost reenrollment or increase credit accumulation. However, this is not the final report on the demonstration; in 2012, NCPR will release a report that synthesizes the findings across all of the colleges studied and includes an additional semester of student follow-up at each college. (Contains 1 figure and 9 footnotes.) [This paper was written with Amanda Grossman. For the full report, "Learning Communities for Students in Developmental English: Impact Studies at Merced College and the Community College of Baltimore County," see ED529251.]

In 2004, Lumina Foundation for Education launched "Achieving the Dream: Community Colleges Count," a national initiative aimed at improving success among community college students, particularly low-income students and students of color. Now encompassing more than 130 institutions in 24 states and the District of Columbia, Achieving the Dream helps community colleges build a "culture of evidence" by using student records and other data to examine students' performance over time and to identify barriers to academic progress. From there, community colleges are expected to develop intervention strategies designed to improve student outcomes; conduct further research on student progress; and bring effective programs to scale. As a result, it is anticipated that colleges will see measurable improvements over time in student outcomes, including increased progress through developmental education and college-level "gatekeeper" (introductory) courses, grades, persistence, and completion of credentials. This report examines the first 26 colleges to join Achieving the Dream in 2004 (known as the "Round 1" colleges), and tracks their progress through spring 2009. The key findings are: (1) Four out of five Round 1 colleges adopted practices associated with a moderate to strong culture of evidence; (2) Colleges that made the greatest strides shared several key characteristics; (3) Colleges instituted a wide range of strategies to improve student achievement, but a majority of them remained small in scale; (4) Achieving the Dream had an important influence on most colleges; and (5) Trends in student outcomes remained relatively unchanged, with a few exceptions. (Contains 1 figure, 2 tables, 1 box, and 3 footnotes.) [For the full report, see ED516014.]

In 2004, Lumina Foundation for Education launched "Achieving the Dream: Community Colleges Count," a national initiative aimed at improving success among community college students, particularly low-income students and students of color. Now encompassing more than 130 institutions in 24 states and the District of Columbia, Achieving the Dream helps community colleges build a "culture of evidence" by using student records and other data to examine students' performance over time and to identify barriers to academic progress. From there, community colleges are expected to develop intervention strategies designed to improve student outcomes; conduct further research on student progress; and bring effective programs to scale. As a result, it is anticipated that colleges will see measurable improvements over time in student outcomes, including increased progress through developmental education and college-level "gatekeeper" (introductory) courses, grades, persistence, and completion of credentials. This report examines the first 26 colleges to join Achieving the Dream in 2004 (known as the "Round 1" colleges), and tracks their progress through spring 2009. The key findings are: (1) Four out of five Round 1 colleges adopted practices associated with a moderate to strong culture of evidence; (2) Colleges that made the greatest strides shared several key characteristics; (3) Colleges instituted a wide range of strategies to improve student achievement, but a majority of them remained small in scale; (4) Achieving the Dream had an important influence on most colleges; and (5) Trends in student outcomes remained relatively unchanged, with a few exceptions. Appendices include: (1) Methodology for Survey of Faculty and Administrators at Achieving the Dream Colleges; and (2) Technical Appendix. Individual chapters contain footnotes. (Contains 11 tables, 29 figures, and 16 boxes.) [For the executive summary, see ED516015.]