Search

In fall 2016, CAPR began a randomized controlled trial of multiple measures assessment (MMA) in community colleges in the State University of New York (SUNY) to learn whether MMA yields placement determinations that lead to better student outcomes than a system based on test scores alone. In 2020, a report was released on students' outcomes after three terms. CAPR then launched a follow-up study to estimate outcomes for a longer time period. In this follow-up study, student outcomes, including college-level math and English enrollment and completion and college-level credit attainment, were tracked for at least nine terms from the time of testing, through spring 2021. The findings were disaggregated by race/ethnicity, Pell recipient status, and gender subgroups; a cost analysis of MMA was also conducted. This brief focuses on "bump-zone" findings--those for the subset of students whose placements changed (or would have changed) under MMA. The study finds that--four and a half years after random assignment--students who were "bumped up" into college-level math and English courses through MMA were much more likely to have enrolled in and completed a college-level course (with a grade of C or higher) than similar business-as-usual group students. The benefits of MMA were likely driven primarily by increased access to college-level courses rather than by any improved accuracy from using MMA. Regardless of subject area, program group students who were bumped up through MMA had better outcomes than similar students in the business-as-usual group, and program group students who were bumped down through MMA had worse outcomes than similar business-as-usual group students. It is important to recognize that MMA can be designed in a way that promotes more access and that prevents students from receiving a lower placement. [For the accompanying working paper, see ED632523.]

Multiple measures assessment (MMA) has gained considerable momentum over the past decade as an alternative to traditional test-based procedures for placing incoming students into developmental or college-level coursework in math and English at broad-access colleges. Compared to standardized tests, which measure student performance at a single point in time, MMA (which often emphasizes high school GPA as a measure) provides a more holistic picture of students' academic preparation. Despite positive impacts on student outcomes that have been found by recent research on MMA, questions remain about whether the positive effects of MMA are sustained over time. This study--a follow-up to prior research using the same sample of students--employs a randomized controlled trial to investigate whether algorithmic MMA placement used at seven State University of New York (SUNY) community colleges led to better student outcomes, for up to four and a half years after randomization, than a system based on test scores alone. Nearly 13,000 incoming students who arrived at the seven colleges in fall 2016, spring 2017, and fall 2017 took placement tests and were randomly assigned to be placed using either the status quo method (business-as-usual group) or the alternative, algorithmic MMA method (program group). Using this sample, we estimate the overall treatment effects on placement into, enrollment in, and completion of college-level math and English as well as effects on other outcomes. We conduct similar analyses on race/ethnicity, Pell recipient status, and gender subgroups. We also descriptively examine the proportion of program group students who were bumped up (i.e., their placement changed from a developmental course placement to a college-level course placement) and bumped down (i.e., their placement changed from a college-level course placement to a developmental course placement) by the MMA algorithm, and we perform a cost-effectiveness analysis. We find that the MMA method used at the colleges improved access to and success in college-level courses and that lower cut scores in English rather than math are associated with larger and longer lasting impacts on completion of college-level coursework. While MMA improved outcomes among student subgroups, it had little to no impact on gaps in outcomes between subgroups. We also find that bumped-up students had substantially better outcomes in both math and English, while bumped-down students had substantially worse outcomes. Our results suggest that increased access to college-level courses is the driving factor in the positive outcomes experienced by program group students and that placement into standalone developmental courses can have detrimental effects on student outcomes. In the discussion of the study's results, we make recommendations for adopting MMA at colleges. Implemented together with other initiatives to support students, MMA can be a first step on the path to success for incoming students. [For the accompanying research brief, see ED632528.]

This report describes a study of healthcare training programs that CCRC conducted to understand the current role that community colleges play in training healthcare workers, including public health workers. In an analysis of 2019-20 data from the U.S. Department of Education's Integrated Postsecondary Education Data System (IPEDS), the authors examine the range and types of healthcare programs offered by postsecondary institutions and the number and characteristics of students who graduate from these programs. Findings from the study provide policymakers and community college leaders a better understanding of the role community colleges currently play and their potential to take on greater responsibility in training healthcare workers.

Test-only placement systems are associated with inaccurate placement determinations that can perpetuate college achievement gaps by race/ethnicity and socioeconomic status (SES). In response to the pitfalls of traditional, test-only placement systems, colleges across the country are increasingly experimenting with and adopting alternative placement strategies that reduce the number of students assigned to prerequisite developmental education and increase access to college-level courses. These reforms benefit underserved students but by themselves are not enough to eradicate long-standing disparities by race/ethnicity and SES in outcomes such as introductory college-level (or gateway) course completion and credential attainment. For placement reform to be both effective and equitable, it is best coupled with additional related reforms, including improvements to curriculum, instruction, and student supports. Aimed at practitioners, this brief provides guidance to institutions seeking to design and implement placement systems that redress limitations of test-only systems and that work in conjunction with other reforms to generate more equitable outcomes. It draws on research literature as well as examples from the field to highlight promising strategies for addressing barriers to equitable access to and success in college-level courses, including barriers that may persist after broad placement reform has been implemented.

In response to research raising concerns about developmental placement, new placement systems and curricular models designed to increase the number of students placed into entry-level college courses have emerged in recent years. These systems aim to more accurately identify students who may benefit from some type of developmental instruction and those who are ready for entry-level college courses. This brief presents the findings of CAPR's exploratory study of informed self-placement (ISP), a placement system in which colleges provide information about placement policies, available courses, and other relevant topics to engage students as active participants in their own placement. ISP, which is also called guided self-placement or directed self-placement, is of interest because it does not rely solely, if at all, on standardized test scores, which research shows are not reliable predictors of performance in college. Moreover, many colleges adopted ISP during the COVID-19 pandemic, when this study began, because they experienced difficulty administering standardized tests and were seeking placement methods that could be easily used in a virtual setting. The study examined course enrollment and completion trends among students placed using ISP practices and sought to document varied approaches to ISP. The researchers collected demographic and outcome data on students' math and English course enrollments and completions over the last five academic years at three Nevada colleges offering associate and bachelor's degrees. They also interviewed assessment and placement scholars and representatives of higher education systems and institutions currently implementing some form of ISP. This brief provides a taxonomy of various placement systems, shares descriptive data on course enrollment and completion, and identifies important equity and access considerations for states and institutions interested in implementing ISP. Although more research is needed to understand the causal impacts of ISP on student outcomes, the data suggest that ISP has the potential to improve students' access to college-level coursework.

Research suggests that using only standardized placement test scores to determine which new college students should take developmental coursework is inadequate, as many students may be unnecessarily assigned to developmental courses. These courses generally take one or more semesters to complete, delaying entry into college-level coursework, and students earn no credit toward a credential for completing them. MDRC and the Community College Research Center recently completed two experimental studies of multiple measures assessment (MMA), in which colleges use high school GPA and other measures in addition to placement test scores to assign incoming students to either developmental or college-level courses in math and English. Overall, results from both studies indicate that student outcomes improve under MMA as compared to status quo placement based on test scores alone. In this brief, we explain what we have learned from these studies.

Despite evidence suggesting that standardized tests are an imperfect measure of academic readiness, many community colleges across the country rely on them alone to determine student course placement. However, when the COVID-19 pandemic forced campuses to transition to remote operations in spring 2020, proctoring in-person placement tests became all but impossible. Though some institutions may have continued to administer the tests remotely, in many cases, the capacity to do so was significantly reduced. In response, systems and colleges sought out innovative ways to determine whether students need developmental education, with many adopting multiple measures assessment and placement systems that take into account factors such as high school GPA and noncognitive assessment results. Based on interviews with system leaders and college administrators, faculty, and staff, this brief presents four short case studies on how community college state systems changed placement policies and supported new placement practices in the midst of the pandemic. The case studies--which focus on Indiana, Virginia, Texas, and Washington--address common multiple measures assessment implementation challenges such as facilitating buy-in, providing implementation support, combatting initiative fatigue, and establishing data-informed evaluation processes.

In the United States, the prevailing high school mathematics course sequence begins with a year of Algebra I, followed by a year of geometry and a year of Algebra II. Educators and others have raised concerns about the extent to which this sequence, which prioritizes the mastery of algebra, is appropriate for the longer-term education and career goals of students who do not intend to pursue STEM degrees in college. These concerns have impelled educators and policymakers to reexamine the prominence of algebra in high school mathematics curricula and to consider new approaches that provide students with more mathematics course options better aligned with their academic and career goals. In this paper, we explore existing approaches to high school mathematics curricula as well as new developments in the field. To start, we discuss a range of high school mathematics course sequences that are currently offered across the country and look at some of the systemic challenges embedded within the traditional paradigm. Then we explore federal and state changes to the provision of high school mathematics in the early 21st century, which we follow with a look at the influence of postsecondary institutions on high school math curricula. We then introduce short case studies of innovative high school math reforms that are occurring in five states. We conclude the paper by considering the Charles A. Dana Center's new initiative, Launch Years, and how this project works to reimagine high school mathematics and its relationship to postsecondary education and careers.

Virtually all community colleges and more than 90 percent of public four-year colleges use the results of placement tests--either alone or in concert with other information--to determine whether students are ready for college-level coursework or need remedial help in math or English. Evidence suggests that placement tests do a poor job of indicating which students need remediation. The Center for the Analysis of Postsecondary Readiness (CAPR) is studying an alternative placement system that uses multiple measures--including both placement test scores and high school GPAs--in predictive algorithms to place incoming students into remedial or college-level courses. Seven community colleges in the State University of New York system participated in the random assignment study to determine whether multiple measures placement leads to better student outcomes than a system based on test scores alone. Using multiple measures placement, many more students were assigned to college-level courses. In math, gains in college-level enrollment and completion were small and short-lived. But in English, the effects were much larger and lasted through at least three semesters. Regardless of whether they were predicted to succeed, students did better when they were allowed to start in college-level courses. A report on longer-term outcomes from the study will be released in summer 2022.

Community college students invest considerable time and money into attending college, but too few of them complete their programs of study. This paper discusses how the development of an overall framing vision for student success, the implementation of evidence-based practices, and the establishment of a culture that is both committed to student success and conducive to innovation are useful, complementary approaches that colleges can pursue to improve student outcomes.

Until recently, most community colleges have done little through the new student onboarding process to help entering students explore academic and career interests, choose a program of study aligned to their interests, and develop a plan for completing such a program. This has been the case even though many students start college without clear goals or even a good sense of how academic programs are connected to particular career areas. This is Part 2 of a three-part packet designed to provide guidance to colleges seeking to redesign their new student onboarding practices. Part 1 of the packet reviews research on why the conventional community college approach to new student onboarding is often unsuccessful in helping students choose and plan a program; it also describes how some colleges are rethinking the onboarding process as part of larger guided pathways reforms. This second part summarizes students' experiences with, and reactions to, the onboarding process as it occurs at two Florida community colleges that have redesigned their onboarding practices for degree-seeking students: Indian River State College (IRSC) and St. Petersburg College (SPC). The authors organize the discussion around three key onboarding goals: helping students to (1) explore interests and programs of study; (2) gain experiences in a program so as to gain confidence in their selected pathway; and (3) engage in academic and career planning so as to understand the best route to program completion. Part 3 of the packet provides exercises to help colleges plan an onboarding redesign. [For "Redesigning Community College Student Onboarding through Guided Pathways. Part 1: Rethinking New Student Onboarding," see ED604992. Part 3 is not available in ERIC. For Part 3 visit https://ccrc.tc.columbia.edu/publications/redesigning-community-college-onboarding-guided-pathways.html.]

Colleges throughout the United States are evaluating the effectiveness of the strategies used to decide whether to place students into college-level or developmental education courses. Developmental, or remedial, courses are designed to develop the reading, writing, or math skills of students deemed underprepared for college-level courses, a determination usually made through standardized placement tests. However, increasing numbers of colleges are using multiple measures to place students, including additional types of placement tests, high school transcripts, and evaluations of student motivation. The current study was developed to add to the understanding about the implementation, cost, and efficacy of an multiple measures assessment (MMA) system using locally determined rules. As part of a randomized controlled trial, the study team evaluated MMA programs and interviewed and observed staff at five colleges in Minnesota and Wisconsin; it also wrote a short case study about one Wisconsin college. The five colleges in the random assignment study targeted all students taking placement tests in the months before the fall 2018 semester. In the four colleges included in the current analysis, 5,282 students participated in the study; of these, 3,677 were tested for English, and 4,487 were tested for math. The findings suggest that while implementation (especially automation) was not easy, it was possible; and using the new MMA systems became much easier once they were established.

Many incoming college students are referred to remedial programs in math or English based on scores they earn on standardized placement tests. Yet research shows that some students assigned to remediation based on test scores would likely succeed in a college-level course in the same subject area without first taking a remedial course if given that opportunity. Research also suggests that other measures of student skills and performance, and in particular high school grade point average (GPA), may be useful in assessing college readiness. The Center for the Analysis of Postsecondary Readiness (CAPR) is conducting a random assignment study of a multiple measures placement system based on data analytics to determine whether it yields placement determinations that lead to better student outcomes than a system based on test scores alone. Seven community colleges in the State University of New York (SUNY) system are participating in the study. The alternative placement system evaluated uses data on prior students to weight multiple measures--including both placement test scores and high school GPAs--in predictive algorithms developed at each college that are then used to place incoming students into remedial or college-level courses. Over 13,000 incoming students who arrived at these colleges in the fall 2016, spring 2017, and fall 2017 terms were randomly assigned to be placed using either the status quo placement system (the control group) or the alternative placement system (the program group). The three cohorts of students will be tracked through the fall 2018 term, resulting in the collection of three to five semesters of outcomes data, depending on the cohort. This interim report, the first of two, examines implementation of the alternative placement system at the colleges and presents results on first-term impacts for 4,729 students in the fall 2016 cohort. The initial results are promising. The final report, to be released in 2019, will examine a range of student outcomes for all three cohorts, including completion of introductory college-level courses, persistence, and the accumulation of college credits over the long term. [This report was written with Dan Cullinan.]

Guided pathways reforms require colleges to rethink how they teach and support students. The case studies in Part 2 of this practitioner packet examine how colleges at the forefront of these reforms are transforming their programs and support services by implementing three key guided pathways practices: (1) meta-majors (clusters of programs in similar academic and career fields that can help students understand a college's offerings); (2) career exploration; and (3) intensive advising. The three colleges (Cleveland State Community College in Tennessee, Jackson College in Michigan, and Indian River State College in southeast Florida) are participating in the American Association of Community Colleges' Pathways Project, which is supporting more than 40 colleges nationally to implement guided pathways reforms at scale. [For "Part 1: A Reform Moves from Theory to Practice," see ED582819. For "Part 3: Timeline and Tips for Implementing Pathways Reforms," see ED582823.]

A growing number of institutions undertaking "guided pathways" reforms are rethinking how students select programs of study, choose courses, and make other program-related choices. One primary aim is to help students make thoughtful decisions about their programs early on as a means to encourage faster and more satisfying college completion. The City Colleges of Chicago (CCC) has been undertaking a guided pathways reform effort for several years. Based largely on student interviews, this paper describes how degree-seeking students at CCC make choices about their programs in their first year of enrollment, focusing especially on how they interact with advisors and how they use college-based resources in program selection and program planning. We find that the complex process of program selection and planning is undertaken differently by students with greater or lesser tolerance for ambiguity, and that inconsistent or conflicting information about program details is particularly frustrating for some students. We recommend that institutions engaging in guided pathways reform efforts clarify advising processes and materials to reduce contradictory information and confusion.

Community college transfer students encounter challenges progressing toward a bachelor's degree, leading to widespread transfer credit loss. This in turn may lower students' chances of credential completion and increase the time and costs for students, their families, and taxpayers. In this study we review three definitions of credit transfer inefficiency--"credit transferability," "credit applicability," and "excess credits among completers"--focusing on the last to examine why students who start at a community college and transfer to a four-year institution so often end up with excess credits that do not count toward a bachelor's degree. To shed light on credit transfer inefficiency, we examine the course-taking behaviors of community college transfer students who earn bachelor's degrees with numerous excess credits compared with transfer students who earn bachelor's degrees with few excess credits. We employ data-mining techniques to analyze student transcripts from two state systems, enabling us to examine a large number of variables that could explain the variation in students' excess credits at graduation. These variables include not only student demographics but also the types and timing of courses taken. Overall, we find more excess credits associated with several factors, including taking larger proportions of 100- and 200-level courses and smaller proportions of 300-level courses throughout students' progression toward completion, and taking 100-level courses in any subject--and specifically 100-level math courses--immediately after transferring to a four-year institution. Findings suggest that institutions could help students reduce credit transfer inefficiency by encouraging them to explore and choose a bachelor's degree major early on so they can take the required lower division (100- and 200-level) courses at the community college, thereby enabling them to take mostly upper division 300- and 400-level courses in their desired major field once they transfer to a four-year institution.

This paper presents the findings from a follow-up quantitative analysis of the Community College of Baltimore County's Accelerated Learning Program (ALP). The results suggest that among students who enroll in the highest level developmental writing course, participation in ALP is associated with substantially better outcomes in terms of English 101 completion and English 102 completion (college-level English courses), which corroborates the results of a similar analysis completed in 2010. Moreover, using a larger number of cohorts and tracking students over a longer period of time, we also found that ALP students were more likely than non-ALP students to persist to the next year. In addition to specific subgroup analyses, we compared college-ready students enrolled in ALP sections of English 101 with their counterparts in wholly college-ready sections, and found that those in ALP sections had equivalent performance within English 101 itself, but slightly lower subsequent college-level course enrollment and completion. (Contains 2 figures, 8 tables, and 7 footnotes.)