Search

Community colleges provide postsecondary education to millions of students in the United States each year, but their graduation rates are low. Many community colleges have implemented interventions to help students persist in college and earn degrees. MDRC has studied many such interventions; several of them improved students' academic outcomes, but the effects varied. This report synthesizes results from MDRC's studies. The data include findings from 30 studies of 39 interventions at 45 colleges. The interventions included different components designed to help students succeed: enhanced advising; enhanced tutoring; financial support; instructional reform; learning communities (which enroll small groups of students together in two or more courses); promoting full-time enrollment in fall and spring, summer enrollment, or both; and success courses (designed to help new students navigate college and build relevant skills). MDRC conducted a quantitative synthesis of the studies' results to examine the relationship between various intervention features and effects on selected student outcomes. This report draws from the quantitative synthesis and also from detailed information on how the interventions were implemented. The research also points to some areas for further exploration. Those areas include how to support part-time community college students effectively, the implementation and effects of programs targeting students in rural areas and in small colleges, and the potential role of community college interventions in reducing racial inequality in students' academic outcomes.

What works to help community college students progress academically? This brief synthesizes 20 years of rigorous research by MDRC, presenting new evidence about key attributes of community college interventions that are positively related to larger impacts on students' academic progress. Findings: Findings are based on a synthesis of evidence from 30 randomized controlled trials of 39 postsecondary interventions involving 60,000 students. The results of this research consistently indicate that the impacts of community college interventions increase with: (1) The comprehensiveness of the intervention, as measured by its number of components; and (2) The promotion of full-time enrollment (during fall and spring) and summer enrollment. Less consistent, but still promising evidence suggests the impacts of community college interventions also increase with the extent that they increase: (1) Advising use among students; (2) Tutoring use among students; and (3) Financial support for students (though the evidence is least consistent for this component). Consequently, the preceding five intervention features seem like a reasonable, evidence-based place to start when developing community college policy, designing a new community college intervention, or enhancing an existing community college intervention.

Some education programs' early positive effects disappear over time. Other programs have unanticipated positive long-term effects. Foundations warn of the dangers of putting too much weight on in-program effects, which, they say, often fade after a program ends. This Issue Focus tackles the topic of post-program effects in postsecondary education. Are in-program effects--that is, the effects observed while the program was active--maintained once the program ends? Do they grow and improve? Or do they fade out? This investigation capitalizes on two decades of rigorous program evaluations conducted by MDRC, including approximately 25 postsecondary programs, to consider those questions. The Higher Education Randomized Controlled Trials (THE RCT) draw from a database of student-level data from 31 MDRC projects, sampling 67,400 stu­dents. The results of THE RCT are striking: During the year after these programs ended, effects on academic progress were consistently maintained. There is no evidence of discernible fade-out, which provides encouraging information about the lasting value of many postsecondary programs and the value of evidence collected from them. There was also no evidence of post-program growth, perhaps prompting the need to develop programs that equip students for success over the long term.

Most community college students are referred to developmental education courses to build basic skills. These students often struggle in these courses and college more broadly. CUNY Start is a prematriculation program for students assessed as having significant remedial needs. CUNY Start students delay matriculation for one semester and receive time-intensive instruction in math, reading, and writing with a prescribed pedagogy delivered by trained teachers. The program aims to help students complete remediation and prepare for college-level courses. This article describes the results of an experiment at four community colleges (n [is approximately equal to] 3,800). We estimate that over three years, including one semester that students spent in the program and two-and-a-half years after the program was complete, CUNY Start substantially increased college readiness, slightly increased credit accumulation, and modestly increased graduation rates (by increasing participation in CUNY's highly effective ASAP).

Nationwide, only 24 percent of community college students earn a degree or certificate within three years of matriculat­ing. Students who enroll in summer classes at the end of the freshman year are more likely to persist and graduate. Summer enrollment enables students to earn additional credits, reducing the time it takes to earn a degree. Summer enrollment also keeps students engaged in school, which may reduce the likelihood that they will drop out between spring and fall. Despite these potential benefits, only 30 percent of community college students enroll in summer courses, leaving much room for improvement. How can community colleges encourage more students to enroll in summer courses? A prior brief from MDRC's Encouraging Additional Summer Enrollment (EASE) project presented evidence that two interventions, an informational campaign and an informational campaign paired with a "last-dollar" tuition assistance grant (one that covers the gap between financial aid and total tuition and fees), can increase summer enrollment and credit accumulation. While these findings were promising, the interventions were imple­mented and evaluated before a major national policy shift: the reinstatement of year-round federal Pell Grants (or "summer Pell"), which provide additional financial aid for summer enrollment to qualifying students. This brief supplements the prior one in several important ways. First, the EASE project has expanded from 4 colleges and 3,700 students to include 10 Ohio com­munity colleges and over 10,000 students. Second, the interventions have now been implemented in two substantially different policy contexts, enabling an assessment of whether these interventions are robust to the shift in Pell Grant pol­icy. Finally, for the first cohort of study participants it is now possible to determine the interventions' effects on enrollment in the summer of their sophomore year. The positive, statistically significant effects on summer enrollment and credit accumulation persisted after the federal policy change. In addition, a second year of the informational campaign plus tuition assistance (but not the informational campaign alone) increased summer enrollment and credits earned in the sophomore year for the first cohort of students. Neither intervention increased overall enrollment in the fall term. [For the prior brief, "Making Summer Pay Off: Using Behavioral Science to Encourage Postsecondary Summer Enrollment," see ED586221. The Encouraging Additional Summer Enrollment (EASE) project is funded by the Ascendium Education Group.]

Community colleges are a large sector of postsecondary education. In 2016-2017, the United States had nearly 1,000 public 2-year postsecondary institutions (community colleges), serving almost nine million students, representing 39% of all undergraduates. The majority of entering community college students require developmental (or remedial) math. Success rates in the developmental math course sequence and college more broadly are discouragingly low. Policymakers, practitioners, and researchers alike are eagerly searching for reforms to improve success rates, but there is a dearth of causal evidence on the effectiveness of most proposed reforms. We sought to answer the following question: What effect does a modularized, computer-assisted, self-paced approach to developmental math (compared with a more "traditional" direct-instruction course alternative) have on students' likelihood of completing the developmental math course sequence? Findings from a randomized controlled trial (n = 1,403) are presented. The program was well implemented; however, we did not find evidence that this approach was superior to the "traditional" math class. Although these results are disappointing, they are important because modularization and self-paced computer-assisted instruction are popular reforms.

Many students who enter community college are deemed underprepared for college-level courses and are referred to developmental (remedial) education courses to build their math, reading, or writing skills. These students often struggle in developmental courses and in college more broadly. To help them, the City University of New York (CUNY) developed CUNY Start. CUNY Start targets incoming students who are assessed as needing remediation in math, reading, and writing. The program delays college matriculation (enrollment in a degree program) for one semester and provides intensive instruction in math, reading, and writing during that semester with a prescribed instructional approach. It also provides advising, tutoring, and a weekly seminar that teaches students skills they need to succeed in college. This report is an evaluation of the program. Findings in this report include: (1) CUNY Start was implemented as it was designed, and the contrast between the program and the colleges' standard courses and services was substantial; (2) During the first semester in the study, program group students made substantially more progress through developmental education than control group students; effects were especially large in math. In contrast, during that same semester, control group students earned more college credits than program group students, as predicted by CUNY Start's designers; and (3) During the second semester, program group students enrolled at CUNY colleges (that is, participated in CUNY Start or enrolled in any non-CUNY Start courses as matriculated students) at a higher rate than control group students. Seven appendices are included.

Research has shown that students who enroll in summer courses are more likely to persist and graduate. How can postsecondary institutions encourage more students to enroll in summer courses? If more students do enroll, will they experience improved academic outcomes? MDRC's Encouraging Additional Summer Enrollment (EASE) project explores these questions by using insights from behavioral science to encourage more students to enroll in summer courses. In partnership with the Ohio Association of Community Colleges and four community colleges in Ohio, MDRC developed and rigorously evaluated two interventions to encourage summer enrollment: (1) The first intervention was an "informational campaign" of personalized student communications delivered by email and mail that incorporated various behavioral science principles. This campaign aimed to simplify academic, financial aid, and procedural information related to summer enrollment, remind students of that information, and motivate them to register for courses; and (2) The second intervention was a similar "informational campaign paired with a 'last-dollar' tuition-assistance grant" that covered the difference between students' summer tuition and fees and any grant financial aid (such as Pell Grant funding) they had available for summer courses. To test the effectiveness of the two interventions, MDRC randomly assigned first-year, low-income students enrolled in spring 2017 courses to (1) a group that received an informational campaign, (2) a group that received tuition assistance plus an informational campaign, or (3) a control group that received the colleges' standard communications. The study finds that both interventions -- the informational campaign alone and the informational campaign plus tuition assistance -- increased summer enrollment. The informational campaign plus tuition assistance was substantially more effective. Both interventions also had positive, statistically significant effects on credit accumulation, an important indicator of progress toward a degree. These findings demonstrate that improved policies and communication about summer term can improve summer enrollment and, as a result, credit accumulation. The brief concludes with lessons and next steps for the project.

The last two decades have seen a dramatic increase in randomized controlled trials (RCTs) conducted in community colleges. Yet, there is limited empirical information on the design parameters necessary to plan the sample size for RCTs in this context. We provide empirical estimates of key design parameters, discussing lessons based on the pattern of estimates across outcomes, semesters, and studies. Nearly all RCTs in community colleges use student random assignment within blocks (colleges or cohorts). Key design parameters for the minimum detectable true effect (MDTE) are the within-block outcome standard deviation ([sigma][subscript ¦][subscript S]) and the within-block outcome variance explained by baseline covariates like student characteristics (R[superscript 2][subscript ¦][subscript S). The main analyses use student-level data from 8 to 14 RCTs including 5,649-7,099 students (depending on the outcome) with follow-up data for 3 years. A public database created for this paper includes parameter estimates for 30 RCTs including 65,637 students. Enrollment, credits earned, credential attainment, and grade point average. The within-block standard deviation ([sigma]jS) and therefore the MDTE can be much larger in later semesters for enrollment outcomes and cumulative credits earned. There is substantial variation across studies in [sigma][subscript ¦][subscript S] for degree attainment. Baseline covariates explain less than 10% of the variation in student outcomes. When planning the sample size for a study, researchers should be mindful of the follow-up period, use a range of values to calculate the MDTE for outcomes that vary across studies, and assume a value of R[superscript 2][subscript ¦][subscript S] between 0 and 0.05. [This is the online first version of an article published in "Evaluation Review."]

Community colleges nationwide are looking for solutions to help students complete developmental (remedial) math--a known barrier to graduation. Some are offering computer-assisted, modular developmental math courses that allow students to earn credits incrementally and move through the curriculum at their own pace. One of these modularized courses, ModMath, was created at Tarrant County College (TCC) near Fort Worth, Texas. It reorganizes the content of TCC's two semester-long developmental math courses into a set of six modules, each of which is five weeks long. The four primary components of the ModMath intervention are: a diagnostic assessment that places students in a starting module; individual registration into three modules per course section each semester; computer-based instruction delivered online through an instructional software program; and personalized, on-demand assistance in class from an instructor and class aide. MDRC is evaluating ModMath's implementation and its effects on students' academic outcomes using a randomized controlled trial. This report contains implementation findings and some findings on early impacts for the first three semesters of students enrolled in the study: (1) ModMath was well implemented and differed from traditional developmental math courses in both the nature of its instruction and its credit-earning structure; (2) After one semester in the program, students randomly assigned to ModMath (the program group) were, on average, closer to completing the developmental math sequence than were students randomly assigned to traditional, lecture-based courses (the control group). This relatively greater progress was the result of program group students getting credit for completing one or two modules but not the equivalent of an entire course; (3) However, this advantage did not translate into other measures of progress. For example, program group students were not more likely to pass the halfway mark in the developmental math sequence than the control group. More than 70 percent of the students in the study, in either group, were unable to pass this benchmark in the first semester; and (4) ModMath had a small negative effect on the percentage of students who completed the developmental math sequence during their first semester (0.4 percent of program group students compared with 1.9 percent of the control group). While this report contains final findings regarding the implementation of ModMath, it contains only preliminary findings on the program's effects. Data were only available for students who enrolled in the first three semesters of what were ultimately four semesters of enrollment, and the report only follows them for one semester. The final report from this study will draw upon additional data and provide additional evidence about the effect ModMath may have on student outcomes. The following tables containing information from Tarrant County College in Spring 2014, Fall 2014, Spring 2015, and Fall 2015 are appended in Additional Analyses: (1) Student Characteristics at Study Enrollment; (2) TSI Placement Levels of the Program and Control Groups; (3) Student Survey Results; (4) Student Survey Results among Program Group Students; (5) Students Placed Using "MyMathTest" and Other Placement Tests; (6) Early Student Progress in ModMath; (7) Early Math Enrollment and Progress; (8) Early Credits Attempted and Earned; and (9) Early Student Math Progress Under Alternate Assumptions for Students Who Did Not Pass Any Math Classes. [Partners at Tarrant County College (TCC) collaborated on this report.]

The benefits of understanding variation apply on multiple levels. Local policymakers and practitioners need to know both the average impact of an intervention and its variation across settings to properly assess its likely benefits and risks for their jurisdictions. For social scientists, cross-site impact variation offers opportunities to learn about mechanisms or mediators through which interventions produce their impacts and characteristics of settings and sample members that influence or moderate these impacts. And for researchers designing studies, cross-site impact variation can markedly affect the statistical precision of effect estimates and hence influence the sample size requirements for these estimates. One important lesson involves reporting cross-site impact variation and reflects the difference between variation in impact estimates and variation in true impacts. It is a simple matter to produce internally valid estimates of the mean impact of an intervention for each site in a multisite randomized trial. However, reporting cross-site variation in these estimates through a frequency distribution or a standard deviation can greatly overstate the amount of true variation that exists. This can occur because differences between site-specific impact estimates have two sources: (1) true cross-site impact differences and (2) differences in random, site-specific estimation error. For studies without very large site samples, most of the variation in site-specific impact estimates reflects random estimation error. Thus, it is essential to use a rigorous method for inferring the magnitude of true cross-site impact variation.

Community colleges offer a pathway to the middle class for low-income individuals. Although access to college has expanded, graduation rates at community colleges remain low, especially for students who need developmental (remedial) courses to build their math, reading, or writing skills. The City University of New York's (CUNY's) Accelerated Study in Associate Programs (ASAP), launched in 2007, is an uncommonly comprehensive and long-term program designed to help more students graduate and help them graduate more quickly. This report presents results from a random assignment study of ASAP at three CUNY community colleges: Borough of Manhattan, Kingsborough, and LaGuardia. Low-income students who needed one or two developmental courses were randomly assigned either to a program group, who could participate in ASAP, or to a control group, who could receive the usual college services. Comparing the two groups' outcomes provides an estimate of ASAP's effects. Key findings from the report are included. The following are appended: (1) Additional Baseline Information; (2) MDRC Student Survey Documentation and Analyses; and (3) Additional Impact Tables. [See earlier CUNY's ASAP reports: "What Can a Multifaceted Program Do for Community College Students? Early Results from an Evaluation of Accelerated Study in Associate Programs (ASAP) for Developmental Education Students" at ED532840 and "More Graduates: Two-Year Results from an Evaluation of Accelerated Study in Associate Programs (ASAP) for Developmental Education Students" at ED546636.]

In 2010, Hillsborough Community College (HCC), a large multicampus institution in Tampa, Florida, worked with MDRC to create the Mathematics Access Performance Scholarship (MAPS) program to help academically underprepared community college students succeed in developmental math. MAPS provides an incentive for low-income students referred to developmental math to take their math courses early and consecutively, get help with math in an on-campus Math Lab, and strive for passing grades or better, in exchange for a modest scholarship contingent on performance. MAPS is part of the national Performance-Based Scholarship (PBS) Demonstration, which is testing performance-based scholarships with varying student populations and program requirements. MAPS offered eligible students the opportunity to earn a performance-based scholarship of up to $1,800 ($600 per semester for three semesters) for successfully completing a three-course sequence--comprising Beginning Algebra, Intermediate Algebra, and a first-level college math course--within a period of three consecutive semesters plus a summer term. MAPS is the only program in the PBS Demonstration that provides an incentive for a specified course sequence. MDRC used a random assignment design to study the effects of MAPS compared with HCC's standard services. Academic outcomes were tracked for all sample members for two years. The findings at HCC, like those across the PBS Demonstration, are modest but positive: (1) More than 90 percent of program group students earned a MAPS award in the first semester, but scholarship receipt rates declined in the second and third semesters; (2) MAPS students were much more likely to visit a Math Lab than control group students; (3) MAPS helped move students further along in the math course sequence, and MAPS students also accumulated more credits overall -- in part because of their greater progress in math; and (4) The program had no discernible impact on students' retention. Appendices include: (1) Selected Baseline Characteristics, by Research Group; (2) Survey Response Analysis; (3) Impact Tables; and (4) Key Cost Terms.

In many experimental evaluations in the social and medical sciences, individuals are randomly assigned to a treatment arm or a control arm of the experiment. After treatment assignment is determined, individuals within one or both experimental arms are frequently grouped together (e.g., within classrooms or schools, through shared case managers, in group therapy sessions, or through shared doctors) to receive services. Consequently, there may be within-group correlations in outcomes resulting from (1) the process that sorts individuals into groups, (2) service provider effects, and/or (3) peer effects. When estimating the standard error of the impact estimate, it may be necessary to account for within-group correlations in outcomes. This article demonstrates that correlations in outcomes arising from nonrandom sorting of individuals into groups leads to bias in the estimated standard error of the impact estimator reported by common estimation approaches. [This paper was supported by funding from the Judith Gueron Fund for Methodological Innovation in Social Policy Research at MDRC.]

Community colleges play a vital role in higher education, enrolling more than one in every three postsecondary students. While the market share of these institutions has grown over the past 50 years, students' success rates remain low. Consequently, community college stakeholders are searching with mounting urgency for strategies that increase rates of success. We evaluate the effects of one such strategy, learning communities, from a randomized trial of over 1,500 students at a large urban college in the City University of New York (CUNY) system. Findings indicate that the program's positive effects on short-term academic progress (credit accumulation) are maintained seven years after random assignment. In addition, the analysis provides some limited evidence that the program positively affected graduation rates, particularly for those students without remedial English needs, over this period. At the same time, however, there is no discernible evidence that the program improved economic outcomes. This paper concludes by offering sobering reflections on trying to detect the effects of higher education interventions on future earnings.

Empirical evidence confirms that increased education is positively associated with higher earnings across a wide spectrum of fields and student demographics (Barrow & Rouse, 2005; Card, 2001; Carneiro, Heckman, & Vytlacil, 2011; Dadgar & Weiss, 2012; Dynarski, 2008; Jacobson & Mokher, 2009; Jepsen, Troske, & Coomes, 2009; Kane & Rouse, 1995; Marcotte, 2004). College degree holders earn higher salaries and experience unemployment less frequently than those who do not have college degrees (Dadgar & Weiss, 2012), and those who attended college, regardless of degree receipt, report higher rates of job satisfaction, promotion opportunities, increased work responsibilities, and improved work performance (Hoachlander, Sikora, & Horn, 2003). For many low-income students, the pathway to higher education and increased earnings begins in community colleges, which currently enroll about one-third of American degreeseekers (Digest of Education Statistics, 2012). However, community college students' persistence and completion rates leave much room for improvement. Among students beginning at public two-year colleges, fewer than half earn a credential or transfer to a four-year institution within six years after their initial enrollment (Radford, Berkner, Wheeless, & Shepherd, 2010). Many community college students stop or drop out long before graduation day, losing out on the full complement of higher earnings and employment outcomes associated with higher education. One factor contributing to these low success rates is the fact that many students arrive on campus only to find that they are required to take non-credit-bearing courses, called "developmental" or "remedial" education courses, prior to enrolling in college-level courses (Duke & Strawn, 2008). One estimate of the prevalence of students required to complete developmental coursework suggests that among students whose first institution attended was a community college, around 60 percent took a remedial course at a postsecondary institution (Adelman, 2004; Bailey, Jeong, & Cho, 2009). Unsurprisingly, degree or certificate attainment rates among students who need developmental education are even lower than those of the general population (Attewell, Lavin, Domina, & Levey, 2006). Students in need of developmental education are not just failing to earn a credential -- the majority never complete the developmental course sequence to which they are referred (Bailey 2009). One popular strategy aimed at addressing the issues of students referred to developmental courses (as well as students in college-level courses) is called "learning communities," or LCs. The LC evaluated in this study produced positive short-term impacts on academic outcomes including credits attempted and credits earned and students' progression through the developmental reading sequence of courses. Given these positive results, similar learning communities for developmental reading and math students in community colleges were replicated and evaluated at five other schools. Tables and figures are appended.

This policy brief presents results from a random assignment evaluation of the City University of New York's Accelerated Study in Associate Programs (ASAP). An ambitious and promising endeavor, ASAP provides a comprehensive array of services and supports to help community college students graduate and to help them graduate sooner. The evaluation targeted low-income students who needed one or two developmental (remedial) courses. ASAP requires students to enroll full time and provides block-scheduled classes, comprehensive advisement, tutoring, career services, a tuition waiver, free monthly MetroCards for use on public transportation, and free use of textbooks for up to three years. After two years, compared with regular college services, ASAP increased the number of credits students earned as well as their persistence in college. Most notably, the program boosted two-year graduation rates substantially--by 66 percent. A future report will present the program's effects after three years.

This paper presents research that explores similarities and differences across six randomized controlled trials of learning communities in community colleges that were conducted by MDRC and the National Center for Postsecondary Research. Five of these studies track students' progress in the program semester and two follow-up semesters, and one study follows students for six years. These studies provide the most extensive evidence available on the promise and limitations of learning communities for improving the academic outcomes of students in community colleges. The present research examines several competing explanations for mixed findings and draws lessons to inform further research in the field. The following six community colleges were involved in the studies: (1) Community College of Baltimore County (CCBC) in suburban Maryland; (2) Hillsborough Community College in Tampa, Florida; (3) Houston Community College in Houston, Texas; (4) Kingsborough Community College in Brooklyn, New York; (5) Merced College in Merced, California; and (6) Queensborough Community College in Queens, New York. The learning communities each lasted for one semester, and consisted of four key components, although there was variation in their emphasis and implementation: (1) linked courses and student cohorts; (2) faculty collaboration; (3) instructional practices; and (4) student supports. More than 7,000 students were randomly assigned to either the program group or the control group. The researchers used several data sources: (1) the baseline information form; (2) operational site visits, field research, and instructor survey; and (3) student records. Results from the studies suggest the following: (1) One-semester learning communities can have a long-term impact and even boost graduation, as shown in the study of the Kingsborough program; and (2) The combined results of all six trials suggest that, on average, learning communities for developmental education students produce only a modest impact on credits earned in the targeted subject of English or mathematics. The research provides good tests of learning communities as they appear to be typically enacted, but not a test of the "ideal" or "advanced" learning communities described in the literature. Tables and figures are appended. [This report was written with Kelley Fong, Hannah Fresques, and Jedediah Teres.]

To date, evaluation research and policy analysis have focused mainly on average program impacts and paid little systematic attention to their variation. Recently, the growing number of multi-site randomized trials that are being planned and conducted make it increasingly feasible to study "cross-site" variation in impacts. Important technical questions arise when one uses data from a multi-site randomized trial to estimate the magnitude of cross-site impact variation and assess the statistical significance of these estimates. Some of these questions arise from the likelihood that variation in "individual" outcomes (which in the context of multi-level models is often referred to as a "level-one residual variance" or "residual variance" for short) might be different for the treatment group and control group members and/or might be different for different sites. In addition, to the extent that different populations and local conditions are represented by different sites, the variance of individual "counterfactual" outcomes (i.e. potential outcomes under no treatment) and therefore the variance of control group members' outcomes will differ across sites. This situation can be exacerbated if "the extent of" individual variation in program impacts also varies across sites because of their differences client populations and local conditions. These "differences in individual outcome variation" represent forms of heteroscedasticity which the authors refer to as: (1) T/C heteroscedasticity; and (2) cross-site heteroscedasticity. In this study, the following methodological questions are addressed through a series of simulations: To what extent are estimators of cross-site impact variation biased and tests of statistical significance incorrect when one does not take into account how the variance of individual outcomes differs between treatment and control group members and/or across sites? In other words, in practice, what are the risks of "estimating too few" residual variances and how does this risk vary under a wide range of conditions? (2) Among different estimators that take heteroscedasticity into account--estimators based on either a random-effects model, on a mixed-effects model or on no model (a method of moments estimator)--how do they compare? and (3) What recommendations can be made for practice?

Prior to the mandatory testing and reporting required by the No Child Left Behind Act (NCLB), school improvement efforts were shown to lack coherence (Newman, Smith, Allensworth, & Bryk, 2001) and often included conflicting programs (Hatch, 2002). Part of the theory of performance-based accountability in general, and NCLB in particular, was based on the belief that regular and comprehensive evidence would help to focus these efforts. Research seems to indicate, however, that individuals vary widely in their assumptions about the value and purpose of evidence (Coburn & Talbert, 2006), that it is often difficult to identify students' challenges based on the resulting annual data (Black & Wiliam, 1998), and that searches for meaningful remedies to real or imagined problems are often extremely limited, somewhat chaotic, and frequently lead back to familiar practices as opposed to real change or innovation (Gross, Kirst, Holland, & Luschei, 2005). In this CPRE Policy Brief, the authors examine the extent to which the assumptions in the law manifest themselves in the actions that school leaders take. This brief asks and answers the question: How do school leaders--administrators and teachers--respond to the results of state assessment systems and the pressure of performance-based accountability? And do those responses seem to matter to achievement outcomes? Data Collection Information is appended.

In 2006, the National Center for Postsecondary Research, of which is MDRC is a partner, launched a demonstration of one-semester learning community programs at six colleges; five of these programs focused on developmental education. This is the final report from the project and includes findings from analyses that pool data across these five programs as well as the results for developmental education students at a sixth program at Kingsborough Community College, operated earlier under the Opening Doors demonstration. Across the six programs, almost 7,000 students were randomly assigned, about half into 174 learning communities, and tracked for three semesters. Key findings suggest that when compared with business as usual, one-semester learning communities in developmental education, on average, lead to: (1) A modest (half-credit) estimated impact on credits earned in the targeted subject (English or mathematics) but no impact on credits earned outside the targeted subject; (2) A modest (half-credit) estimated impact on total credits earned; and (3) No impact on persistence in college. The developmental education students in the Kingsborough program, which had some different features from the other five programs, including enhanced support services, showed somewhat larger results than the other sites in credits earned in the targeted subject. An MDRC report on the overall Kingsborough learning communities program, which served "both" developmental and college-ready students, shows a positive impact on degree attainment after six years. The graduation effect was driven primarily by students who had placed into college-level English, although there is also evidence that the program had a positive impact on long-term outcomes for students with the greatest developmental needs in English. Together, these evaluations suggest that, while most typical one-semester learning communities for developmental education students are not likely to lead to large effects on students' outcomes, a program with additional supports can have longer-term impacts for developmental students. Appended are: (1) Impact Analyses; (2) Supplementary Exhibits for Chapter 3; (3) Instructor Survey Details; (4) Cost Details; and (5) Supplementary Table for Chapter 5. Individual chapters contain footnotes. (Contains 25 tables, 10 figures and 2 boxes.) [This paper was written with Jedediah Teres and Kelley Fong. For "The Effects of Learning Communities for Students in Developmental Education: A Synthesis of Findings from Six Community Colleges. Executive Summary," see ED533826.]

In recent years, there has been unprecedented national focus on the importance of increasing the stubbornly low graduation rates of community college students. Most reforms that have been tried are short-term and address one or only a few barriers to student success. The City University of New York's (CUNY's) Accelerated Study in Associate Programs (ASAP), launched in 2007 with funding from Mayor Bloomberg's Center for Economic Opportunity (CEO), is an uncommonly multifaceted and long-term program designed to help community college students graduate. ASAP requires students to attend college full time and provides a rich array of supports and incentives for up to three years, with a goal of graduating at least 50 percent of students within three years. This report presents very promising early findings from a random assignment study of ASAP at three CUNY community colleges: Borough of Manhattan, Kingsborough, and LaGuardia. For the study, ASAP targets low-income students who need one or two developmental (remedial) courses to build their reading, writing, or math skills. The study compares ASAP with regular services and classes at the colleges. Key findings include effects on: (1) Full-time enrollment. During the study's first semester, ASAP increased full-time enrollment by 11 percentage points: 96 percent of the students assigned to ASAP enrolled full time, compared with 85 percent of the comparison group; (2) Credits earned and completing developmental coursework. ASAP increased the average number of credits earned during the first semester by 2.1 credits and increased the proportion of students who completed their developmental coursework by the end of that semester by 15 percentage points; and (3) Semester-to-semester retention. ASAP increased the proportion of students who enrolled in college during the second semester by 10 percentage points and increased full-time enrollment that semester by 21 percentage points. ASAP's early effects are larger than the effects of most of the community college programs MDRC has studied previously. ASAP's comprehensive package of financial aid, services, and supports, together with its full-time attendance requirement, has resulted in students taking and passing more credits than they would have otherwise. Future reports will show whether these effects can be sustained--or even grow--as students continue in this comprehensive, three-year program. Selected Characteristics of Sample Members at Baseline, by Research Group are appended. (Contains 9 tables, 2 boxes and 40 footnotes.) [This paper was written with Hannah Fresques. For "What Can a Multifaceted Program Do for Community College Students? Early Results from an Evaluation of Accelerated Study in Associate Programs (ASAP) for Developmental Education Students. Executive Summary," see ED532834.]

Community college stakeholders are searching with increasing urgency for approaches with the potential to improve the success rates for community college students, particularly those in need of developmental education. "Learning communities," which place cohorts of students together in two or more courses for one semester, are a popular instructional reform community colleges are implementing to improve the outcomes of developmental education students, and have previously been associated with positive social, psychological, and academic outcomes in non-experimental and quasi-experimental research (Engstrom & Tinto, 2008; Stefanou & Salisbury-Glennon, 2002; Tinto, 1997; Tinto, 1998; Zhao & Kuh, 2004). This paper synthesizes results from six random assignment evaluations of developmental education learning communities programs. Five of the six programs were part of the National Center for Postsecondary Research's (NCPR) Learning Communities Demonstration, and the sixth was studied as part of MDRC's Opening Doors Demonstration. The primary question addressed in this study is whether learning communities, compared to "business as usual," lead to better educational outcomes for students who are placed into developmental English and math in community colleges. The authors also examine whether the effects of learning communities vary across colleges and subgroups. (Contains 4 figures, 2 tables, and 6 footnotes.)

This paper presents results from a rigorous random assignment study of Learning Communities programs operated at three of six community colleges participating in the National Center for Postsecondary Research's (NCPR) Learning Communities Demonstration. The demonstration's focus is on determining whether Learning Communities are an effective strategy for helping students who have been referred to developmental education. The setting of this research is Hillsborough Community College, Queensborough Community College and Houston Community College. Findings from these three Learning Communities Demonstration sites are as follows: (1) Hillsborough's fairly basic Learning Communities' model did not have a meaningful impact on students' academic success (in terms of likelihood of completing developmental reading, average total credits attempted/earned, and rates of persistence); (2) Both Houston's and Queensborough's Learning Communities programs showed evidence of helping students progress through the developmental math sequence more quickly; and (3) Houston's and Queensborough's Learning Communities programs both did not show evidence of lasting impacts on credit accumulation or retention. (Contains 2 tables.)

This Brief, based on a report of the same title, presents results from a rigorous study of a basic learning communities program operated at Hillsborough Community College. Hillsborough, one of six community colleges participating in the National Center for Postsecondary Research's (NCPR) Learning Communities Demonstration, is a large, urban community college located in Tampa, Florida, a Gulf Coast city on the west coast of Florida. Hillsborough serves around 24,000 students each year, and three of the college's five campuses--Dale Mabry, Ybor City, and Brandon--participated in the Learning Communities Demonstration. This study targeted a particular subset of the student body, first-semester students whose placement test scores required them to take developmental reading courses. The overall result (for the full sample) of this study is that the learning communities program at Hillsborough did not appear to add value above and beyond the college's usual services. [This brief was written with Jed Teres and Emily Schneider. For the full report, "Learning Communities for Students in Developmental Reading: An Impact Study at Hillsborough Community College", see ED510961.]

Over the last four decades, community colleges have played an increasingly important role in higher education. Today, community colleges enroll more than one in every three undergraduates nationally. Unfortunately, among students who enroll in community colleges with the intent to earn a credential or transfer to a four-year institution, only 51 percent achieve that goal within six years. Many postsecondary institutions operate "learning communities" to improve low rates of success. Basic learning communities simply co-enroll a cohort of students into two classes together. More comprehensive learning communities include additional components: The courses have integrated curricula, instructors collaborate closely, and student services such as enhanced advising and tutoring can be embedded, among other approaches. This executive summary report presents results from a rigorous random assignment study of a basic learning community program at Hillsborough Community College in Tampa Bay, Florida. Hillsborough is one of six community colleges participating in the National Center for Postsecondary Research's Learning Communities Demonstration. The demonstration's focus is on determining whether learning communities are an effective strategy for helping students who need developmental education. Hillsborough's learning communities co-enrolled groups of around 20 students into a developmental reading course and a "college success" course. Three cohorts of students (fall 2007, spring 2008, and fall 2008) participated in the study, for a total of 1,071. The findings show that: (1) The most salient feature of the learning communities implemented at Hillsborough was the co-enrollment of students into linked courses, creating student cohorts; (2) The learning communities at Hillsborough became more comprehensive over the course of the study. In particular, curricular integration and faculty collaboration were generally minimal at the start of the study, but increased over time; (3) Overall (for the full study sample), Hillsborough's learning communities program did not have a meaningful impact on students' academic success; and (4) Corresponding to the maturation of the learning communities program, evidence suggests that the program had positive impacts on some educational outcomes for the third (fall 2008) cohort of students. These results represent the first in a series of impact findings from the Learning Communities Demonstration. Results from the other five demonstration sites will be released in the next several years, providing a rich body of experimental research on the effectiveness of various learning community models in the community college setting. (Contains 4 footnotes.) [This paper was written with Jed Teres and Emily Schneider. For "Learning Communities for Students in Developmental Reading: An Impact Study at Hillsborough Community College," see ED510961.]

Randomized experiments have become an increasingly popular design to evaluate the effectiveness of interventions in education (Spybrook, 2008). Many of the interventions evaluated in education are delivered to groups of students, rather than to individuals. Experiments designed to evaluate programs delivered at the group level often randomize intact groups, such as classes or schools, to treatment and control conditions in order to obtain unbiased estimates of intervention effects (H. S. Bloom, 2006; H. S. Bloom, et al., 2008). A growing body of research has discussed the analytics of such group-randomized trials (GRTs), or cluster-randomized trials, as well as the appropriate interpretation of impact estimates (H. S. Bloom, 2006; H. S. Bloom, Richburg-Hayes, & Black, 2007; H. S. Bloom, et al., 2008; Hedges, 2007; Hedges & Hedberg, 2007; Konstantopoulos, 2008a, 2008b; Spybrook, 2008). This work describes how, in studies of classroom-level interventions that do not randomize teachers to experimental conditions, it will be unclear whether measured differences between program and control group students are a result of the core components of the intervention or the teachers (i.e., teacher effects). This potential confounding is a major concern if teachers are sorted into experimental conditions in such a way that they differ on variable(s) that are related to their effectiveness. This work attempts to make clear the correct interpretation of typically calculated "program impacts" in this situation. In addition, using the magnitude of estimated teacher effects from prior research, this work demonstrates that if teachers are not randomly assigned to experimental conditions, then it is significantly more difficult to establish whether the intervention "works" or if the types of teachers selected to teach in intervention classrooms are simply more/less effective than their control group counterparts. The implications may be quite serious in terms of the usefulness of such studies' findings. (Contains 1 table, 2 figures and 9 footnotes.