Search

In fall 2016, CAPR began a randomized controlled trial of multiple measures assessment (MMA) in community colleges in the State University of New York (SUNY) to learn whether MMA yields placement determinations that lead to better student outcomes than a system based on test scores alone. In 2020, a report was released on students' outcomes after three terms. CAPR then launched a follow-up study to estimate outcomes for a longer time period. In this follow-up study, student outcomes, including college-level math and English enrollment and completion and college-level credit attainment, were tracked for at least nine terms from the time of testing, through spring 2021. The findings were disaggregated by race/ethnicity, Pell recipient status, and gender subgroups; a cost analysis of MMA was also conducted. This brief focuses on "bump-zone" findings--those for the subset of students whose placements changed (or would have changed) under MMA. The study finds that--four and a half years after random assignment--students who were "bumped up" into college-level math and English courses through MMA were much more likely to have enrolled in and completed a college-level course (with a grade of C or higher) than similar business-as-usual group students. The benefits of MMA were likely driven primarily by increased access to college-level courses rather than by any improved accuracy from using MMA. Regardless of subject area, program group students who were bumped up through MMA had better outcomes than similar students in the business-as-usual group, and program group students who were bumped down through MMA had worse outcomes than similar business-as-usual group students. It is important to recognize that MMA can be designed in a way that promotes more access and that prevents students from receiving a lower placement. [For the accompanying working paper, see ED632523.]

Multiple measures assessment (MMA) has gained considerable momentum over the past decade as an alternative to traditional test-based procedures for placing incoming students into developmental or college-level coursework in math and English at broad-access colleges. Compared to standardized tests, which measure student performance at a single point in time, MMA (which often emphasizes high school GPA as a measure) provides a more holistic picture of students' academic preparation. Despite positive impacts on student outcomes that have been found by recent research on MMA, questions remain about whether the positive effects of MMA are sustained over time. This study--a follow-up to prior research using the same sample of students--employs a randomized controlled trial to investigate whether algorithmic MMA placement used at seven State University of New York (SUNY) community colleges led to better student outcomes, for up to four and a half years after randomization, than a system based on test scores alone. Nearly 13,000 incoming students who arrived at the seven colleges in fall 2016, spring 2017, and fall 2017 took placement tests and were randomly assigned to be placed using either the status quo method (business-as-usual group) or the alternative, algorithmic MMA method (program group). Using this sample, we estimate the overall treatment effects on placement into, enrollment in, and completion of college-level math and English as well as effects on other outcomes. We conduct similar analyses on race/ethnicity, Pell recipient status, and gender subgroups. We also descriptively examine the proportion of program group students who were bumped up (i.e., their placement changed from a developmental course placement to a college-level course placement) and bumped down (i.e., their placement changed from a college-level course placement to a developmental course placement) by the MMA algorithm, and we perform a cost-effectiveness analysis. We find that the MMA method used at the colleges improved access to and success in college-level courses and that lower cut scores in English rather than math are associated with larger and longer lasting impacts on completion of college-level coursework. While MMA improved outcomes among student subgroups, it had little to no impact on gaps in outcomes between subgroups. We also find that bumped-up students had substantially better outcomes in both math and English, while bumped-down students had substantially worse outcomes. Our results suggest that increased access to college-level courses is the driving factor in the positive outcomes experienced by program group students and that placement into standalone developmental courses can have detrimental effects on student outcomes. In the discussion of the study's results, we make recommendations for adopting MMA at colleges. Implemented together with other initiatives to support students, MMA can be a first step on the path to success for incoming students. [For the accompanying research brief, see ED632528.]

This report describes a study of healthcare training programs that CCRC conducted to understand the current role that community colleges play in training healthcare workers, including public health workers. In an analysis of 2019-20 data from the U.S. Department of Education's Integrated Postsecondary Education Data System (IPEDS), the authors examine the range and types of healthcare programs offered by postsecondary institutions and the number and characteristics of students who graduate from these programs. Findings from the study provide policymakers and community college leaders a better understanding of the role community colleges currently play and their potential to take on greater responsibility in training healthcare workers.

Test-only placement systems are associated with inaccurate placement determinations that can perpetuate college achievement gaps by race/ethnicity and socioeconomic status (SES). In response to the pitfalls of traditional, test-only placement systems, colleges across the country are increasingly experimenting with and adopting alternative placement strategies that reduce the number of students assigned to prerequisite developmental education and increase access to college-level courses. These reforms benefit underserved students but by themselves are not enough to eradicate long-standing disparities by race/ethnicity and SES in outcomes such as introductory college-level (or gateway) course completion and credential attainment. For placement reform to be both effective and equitable, it is best coupled with additional related reforms, including improvements to curriculum, instruction, and student supports. Aimed at practitioners, this brief provides guidance to institutions seeking to design and implement placement systems that redress limitations of test-only systems and that work in conjunction with other reforms to generate more equitable outcomes. It draws on research literature as well as examples from the field to highlight promising strategies for addressing barriers to equitable access to and success in college-level courses, including barriers that may persist after broad placement reform has been implemented.

In response to research raising concerns about developmental placement, new placement systems and curricular models designed to increase the number of students placed into entry-level college courses have emerged in recent years. These systems aim to more accurately identify students who may benefit from some type of developmental instruction and those who are ready for entry-level college courses. This brief presents the findings of CAPR's exploratory study of informed self-placement (ISP), a placement system in which colleges provide information about placement policies, available courses, and other relevant topics to engage students as active participants in their own placement. ISP, which is also called guided self-placement or directed self-placement, is of interest because it does not rely solely, if at all, on standardized test scores, which research shows are not reliable predictors of performance in college. Moreover, many colleges adopted ISP during the COVID-19 pandemic, when this study began, because they experienced difficulty administering standardized tests and were seeking placement methods that could be easily used in a virtual setting. The study examined course enrollment and completion trends among students placed using ISP practices and sought to document varied approaches to ISP. The researchers collected demographic and outcome data on students' math and English course enrollments and completions over the last five academic years at three Nevada colleges offering associate and bachelor's degrees. They also interviewed assessment and placement scholars and representatives of higher education systems and institutions currently implementing some form of ISP. This brief provides a taxonomy of various placement systems, shares descriptive data on course enrollment and completion, and identifies important equity and access considerations for states and institutions interested in implementing ISP. Although more research is needed to understand the causal impacts of ISP on student outcomes, the data suggest that ISP has the potential to improve students' access to college-level coursework.

Research suggests that using only standardized placement test scores to determine which new college students should take developmental coursework is inadequate, as many students may be unnecessarily assigned to developmental courses. These courses generally take one or more semesters to complete, delaying entry into college-level coursework, and students earn no credit toward a credential for completing them. MDRC and the Community College Research Center recently completed two experimental studies of multiple measures assessment (MMA), in which colleges use high school GPA and other measures in addition to placement test scores to assign incoming students to either developmental or college-level courses in math and English. Overall, results from both studies indicate that student outcomes improve under MMA as compared to status quo placement based on test scores alone. In this brief, we explain what we have learned from these studies.