Search

There is a growing interest in ensuring that students are prepared for college and a career when they finish high school. Research identified several indicators that are associated with college preparedness including attendance, grade point average (GPA), and test scores. However, it is difficult to use these indicators to determine the preparedness level of the nation's students in general. This study used results from National Assessment of Educational Progress (NAEP) mathematics assessment, the "gold standard" assessing what students know and can do in grade 12 to establish college preparedness benchmarks. This study linked ninth-grade student background data and school-reported high school and postsecondary transcript data from the National High School Longitudinal Study of 2009 (HSLS:09) to student item responses on the 2013 National Assessment of Educational Progress (NAEP) mathematics assessment to examine the relationship between college preparedness and NAEP mathematics achievement. NAEP scale scores imputed for all HSLS:09 participants via marginal maximum likelihood regression analyses were used to predict college preparedness. This study extended the earlier investigations of college preparedness based on NAEP grade 12 mathematics data from the prediction of college enrollment alone to the prediction of additional postsecondary outcomes including remedial course-taking and first-year college GPA based on postsecondary transcript data. Defining college preparedness as enrolling in a 2- or 4-year college without remedial course-taking and acquiring a first-year college GPA of 2.7 or higher as did National Assessment Governing Board, this study showed that the probability of college preparedness increased from 26 percent for students performing at the NAEP Basic level to 58 percent at the NAEP Proficient level and 87 percent at the NAEP Advanced level. The probabilities at preparedness increased when the definition of college preparedness omitted remedial course-taking. Although similar patterns were observed for various student groups including race/ethnicity, gender, and socioeconomic status, there were some differences in results across groups. Since the report shows the probability of college preparedness (under various definitions) at each NAEP achievement level, it is thereby also providing important validity information for these NAEP achievement levels.

This paper summarises the exploratory quantitative analysis undertaken to investigate how vocational education and training (VET) students cluster and segment in the Australian VET market. This analysis is outlined in three sections. The first section focuses on 'clustering' as a technique for grouping data and the three clustering algorithms used. These are then discussed in more detail to provide some insights into how they operate. Their specific data requirements, along with their strengths and weaknesses, are also considered. In next section the outputs of the clustering approaches are considered. The resultant clusters are examined to better understand them, and meaningfully label and group them into segments. The final section returns to the raw data. This step was necessary to further explore (in this case, only some of) the identified market segments. Three key market segments are explored: students in targeted English programs; students in social inclusion programs; and migrant students.

Science, technology, engineering, and mathematics (STEM) professional societies (ProS) are uniquely positioned to foster national-level diversity, equity, and inclusion (DEI) reform. ProS serve broad memberships, define disciplinary norms and culture, and inform accrediting bodies, thus providing "excellent leverage with which to design and promote change" (National Academy of Sciences et al., 2005). ProS could be instrumental in achieving the DEI culture reform necessary to optimize engagement of all STEM talent by leveraging disciplinary excellence resulting from diverse teams. Inclusive STEM culture reform requires that underlying "mental models'' be examined (Kania et al., 2018). The "Inclusive Professional Framework for Societies (IPF:Societies)" can help ProS change leaders (i.e., "boundary spanners'") and organizations identify and address mental models hindering DEI reform. "IPF:Societies" uses four "I's"--Identity awareness and Intercultural mindfulness (i.e., equity mindset) on which Inclusive relationships and Influential DEI actions are scaffolded. We discuss how "IPF:Societies" complements existing DEI tools. We explain how "IPF:Societies" can be applied to existing ProS policy and practice associated with common ProS functions (e.g., leadership, membership, conferences, awards, and professional development). Next steps are to pilot "IPF:Societies" with a cohort of STEM ProS. Ultimately, "IPF:Societies" has potential to promote more efficient, effective, and lasting DEI organizational transformation, and contribute to inclusive STEM disciplinary excellence.

Among community college leaders and others interested in reforms to improve student success, there is growing interest in adopting machine learning (ML) techniques to predict credential completion. However, ML algorithms are often complex and are not readily accessible to practitioners for whom a simpler set of near-term measures may serve as sufficient predictors. This study compares the out-of-sample predictive power of early momentum metrics (EMMs)--13 near-term success measures suggested by the literature--with that of metrics from ML-based models that employ approximately 500 predictors for community college credential completion. Using transcript data from approximately 50,000 students at more than 30 community colleges in two states, I find that the EMMs that were modeled by logistic regression accurately predict completion for approximately 80% of students. This classification performance is comparable to that of the ML-based models. The EMMs even outperform the ML-based models in probability estimation. These findings suggest that EMMs are useful predictors for credential completion and that the marginal gain from using an ML-based model over EMMs is small for credential completion prediction when additional predictors do not have strong rationales to be included in an ML-based model, no matter how large the number of those predictors may be.

School value-added models are increasingly used to measure schools' contributions to student success. At the same time, policymakers and researchers agree that schools should support students' socialemotional learning (SEL) as well as academic development. Yet, the evidence regarding whether schools can influence SEL and whether statistical growth models can appropriately measure this influence is limited. Recent work shows meaningful differences across schools in changes in SEL scores by grade (Loeb, Christian, Hough, Meyer, Rice, & West, 2019), but whether these differences represent the effects of schools is still unclear. The current paper builds upon this earlier work by examining the stability of the estimated school-by-grade effects on SEL across two years, using a large-scale SEL survey administered in California's CORE districts. We find that correlations among school effects in the same grades across different years are positive, but they are lower than those for math and English Language Arts (ELA). Schools in the top or the bottom of the school effect distribution are more persistent in their impacts across years than those in the middle of the distribution. Overall, the results provide evidence that these school effects measure real contributions to SEL. However, the low stability of effects from one year to the next draw into question whether including these school value-added measures of self-reported SEL in school performance frameworks and systems would be beneficial.

Principals are widely seen as a key influence on the educational environment of schools, and nearly all principals have experience as teachers. Yet there is no evidence on whether we can predict the effectiveness of principals (as measured by their value added) based on their value added as teachers, an issue we explore using administrative data from Washington. Several descriptive features of the principal labor market stand out. First, teachers who become principals tend to have higher levels of educational attainment while teaching and are less likely to be female, but we find no significant differences in licensure test scores between those teachers who become principals and those we do not observe in the principalship. Second, principal labor markets appear to be quite localized: about 50 percent of principals previously taught in the same district in which they assumed a principalship. We find positive correlations between teacher and principal value added in reading (ELA) and similarly sized but less precise estimates in math. Teachers who become principals have slightly higher teacher value added, but the difference between the two groups is not statistically significant, suggesting that principals are not systematically selected based on their prior effectiveness when serving as a classroom teacher.

In the first systematic study of gender achievement gaps in U.S. school districts, we estimate male-female test score gaps in math and English Language Arts (ELA) for nearly 10,000 school districts in the U.S. We use state accountability test data from third through eighth grade students in the 2008-09 through 2014-15 school years. The average school district in our sample has no gender achievement gap in math, but a gap of roughly 0.23 standard deviations in ELA that favors girls. Both math and ELA gender achievement gaps vary among school districts and are positively correlated -- some districts have more male-favoring gaps and some more female-favoring gaps. We find that math gaps tend to favor males more in socioeconomically advantaged school districts and in districts with larger gender disparities in adult socioeconomic status. These two variables explain about one fifth of the variation in the math gaps. However, we find little or no association between the ELA gender gap and either socioeconomic variable, and we explain virtually none of the geographic variation in ELA gaps.

Nearly two dozen states now administer online exams. These tests have real consequences: their results feed into accountability systems, which have been used for more than a decade to hold schools and districts accountable for their students' learning. We examine the rollout of computer-based testing in Massachusetts over 2 years to investigate test mode effects. Crucial to the study design is the fact that the state administers the same exam (PARCC) in online and offline formats each year during the transitional period. We find mode effects of about 0.10 standard deviations in math and 0.25 standard deviations in English language arts (ELA). Our preferred estimates of the effects of online testing in the second year of administration suggest that mode effects for second-time test takers were about one third as large as the first year in math and about half as large in ELA. There is little evidence of systematic variation in mode effects by student demographic groups, although on ELA tests they are larger for students scoring at the bottom of the achievement distribution.

The Common Core State Standards for Mathematics (CCSS-M) set rigorous standards for each of grades 6, 7 and 8. Strategic Education Research Partnership (SERP) has been working with two school districts, Oakland Unified School District and San Francisco Unified School District, to evaluate extant policies and practices and formulate new policies based on the CCSS-M shifts that promise to work better for students. This work has culminated in school board adoptions of new policies in both districts. This report focuses on the problems, solutions and designs that that have been found in the CCSS-M and can be of immediate use to other districts and states.

This paper presents part of the results of a completed study entitled "A Longitudinal Study of Minority Ph.D.s from 1980-1990: Progress and Outcomes in Science and Engineering at the University of California during Graduate School and Professional Life." It focuses particularly on the graduate school experience and degree of preparation for the professoriate of African American doctoral students in the sciences and engineering, and presents the results of a survey of 33 African American STEM Ph.D.s from the University of California earned between 1980-1990. Relationships with thesis advisors and principal investigators are evaluated by the study participants in fifteen specific areas from highly-ranked intellectual development to low-ranked training in grant writing. Deficits in training and socialization are discussed along with the tension between being both an African American and a graduate student. Career choices and outcomes are presented. These findings, in conjunction with current analyses of graduate education in STEM, suggest ways in which graduate training for all could be improved. (Contains 1 table.) [Financial support for this project was provided by the U.C. Industry-University Cooperative Research Program; and the U.C. Berkeley Graduate Division.]

This report investigates enrollment patterns and academic outcomes in online, hybrid, and face-to-face courses among students who enrolled in Washington State community and technical colleges in the fall of 2004. Students were tracked for nearly five years, until the spring of 2009. Results were similar to those found in a parallel study in Virginia (Jaggars & Xu, 2010). Students who were employed for more hours and students who had demographic characteristics associated with stronger academic preparation were more likely to enroll in online courses; however, students who enrolled in hybrid courses were quite similar to those who enrolled in a purely face-to-face curriculum. After controlling for student characteristics using multilevel regression techniques, results indicated that students were more likely to fail or withdraw from online courses than from face-to-face courses. In addition, students who took online coursework in early terms were slightly but significantly less likely to return to school in subsequent terms, and students who took a higher proportion of credits online were slightly but significantly less likely to attain an educational award or transfer to a four-year institution. In contrast, students were equally likely to complete a hybrid course as to complete a face-to-face course. Additional analyses with a new cohort of students entering in 2008 showed short-term results consistent with those of the 2004 cohort. Given the importance of online learning in terms of student convenience and institutional flexibility, current system supports for online learning should be bolstered and strengthened in order to improve completion rates among online learners. Specific recommendations are discussed in the report's conclusion. (Contains 9 tables, 4 figures and 23 footnotes.)

This paper reports on the Macromedia Accessibility Project (MAP), a collaborative effort developed by Macromedia, Inc., The League for Innovation in the Community College, and Pima Community College, Arizona, in order to explore the questions and challenges of implementing accessibility standards in online developmental math courses in community colleges. The purpose of the project, as a two-part process, was to evaluate the capacity to create online mathematical content in an accessible format, and to identify the differences between updating content and design differences for Web-based courses versus creating new content and design features for Web-based delivery. Project team members were charged to audit and upgrade one existing online math course and create one new online math course. As a summer project commencing June 15, 2002 and ending September 15, 2002, the 3-month timeline called for inclusion of training sessions, groups meetings, and follow-up communications. Comments from MAP team developers were summarized for review. This paper describes the two different processes teams went through in the course revisions and the new course development. The MAP team members completed all project goals and objectives as designated, yet noted that their work barely scratched the surface. Includes 5 figures and recommendations for further research. (NB)

This paper is a report of a study that investigated whether increased private-school competition results in enhanced performance of public schools. Data for the study came from a pooled data set from Georgia Department of Education documents and directories, which included socioeconomic data on students and test scores from Georgia school systems between 1980 and 1990 for 3rd-grade and 10th-grade reading and math tests. The data allowed researchers to address directly the problem of endogeneity. Following an introduction, the report presents a framework of how private competition can affect public-school performance and investigates the conditions for a beneficial competitive effect. The next section explains the empirical models and details the data used in the analysis. The next section presents the empirical results. The empirical results do not lend support to the hypothesis that private-school competition improves public-school performance, as measured by student-exam results. Supplementing the text are five tables that detail test scores and socioeconomic data. (Contains 23 references.) (WFA)

Tracking is widespread in U.S. education. In post-secondary education alone, at least 71% of colleges use a test to track students. However, there are concerns that the most frequently used college placement exams lack validity and reliability, and unnecessarily place students from under-represented groups into remedial courses. While recent research has shown that tracking can have positive effects on student learning, inaccurate placement has consequences: students face misaligned curricula and must pay tuition for remedial courses that do not bear credits toward graduation. We develop an alternative system to place students that uses predictive analytics to combine multiple measures into a placement instrument. Compared to colleges' existing placement tests, the algorithm is more predictive of future performance. We then conduct an experiment across seven colleges to evaluate the algorithm's effects on students. Placement rates into college-level courses increased substantially without reducing pass rates. Adjusting for multiple testing, algorithmic placement generally, though not always, narrowed gaps in college placement rates and remedial course taking across demographic groups. A detailed cost analysis shows that the algorithmic placement system is socially efficient: it saves costs for students while increasing college credits earned, which more than offsets increased costs for colleges. Costs could be reduced with improved data digitization, as opposed to entering data by hand.

There are a number of issues to consider in choosing tests, including alignment of tests and standards, the integration of tests with curriculum and instruction, the quality of the tests, and a clear definition of the purpose of the test. To address the various needs reflected by these issues, PLATO Learning, Inc., offers two curriculum-wide testing systems: NetSchools Orion GATE and PLATO (registered) LINK. For practice in preparation for high-stakes tests, PLATO Learning offers the Simulated Tests in mathematics, reading, and writing. To support needs for placement, progress control, and cumulative testing when using PLATO (registered) courseware, PLATO Learning offers the FASTRACK and Skills Inventory systems, module mastery tests, and course-level assessments. Each of these systems has different characteristics and is designed to serve different needs. Choosing among them involves answering 12 key questions about testing needs, which are provided in this document. (Author/SLD)

This report describes the results from an exploratory project conducted for the National Center for Education Statistics. The purpose of the project was to develop and field test questionnaire items and related methods designed to capture information about the instructional processes used nationally in 8th- to 12th-grade mathematics classrooms. Data from classroom observers, short logs of daily classroom practice, and a teacher questionnaire were used to assess the quality of the items. A survey questionnaire, field tested with 100 teachers, was completed by approximately 300 mathematics teachers, and a case study was conducted with 41 teachers in similar settings with comparable responsibilities. Results of these studies were used to make 10 recommendations to improve the field test validation process and the survey items. Thirteen appendixes contain supplemental information about the study and the test items. Appendix M contains a summary, in chart form, of recommended changes to the field test items. (Contains 21 references.) (SLD)

Grade retention and early dropout are two of the biggest challenges facing education systems in middle-income countries today, representing waste in school resources. We investigate whether reducing parent-school information gaps can improve outcomes that are early-warning signals for grade retention and dropout. We conducted an experiment in low-income schools in Chile to test the effects and behavioral changes triggered by a program that sends attendance, grade, and classroom behavior information to parents via weekly and monthly text messages. Our 18-month intervention raised average math GPA by 0.09 of a standard deviation and increased the share of students satisfying attendance requirements for grade promotion by 4.5 percentage points. Treatment effects were larger for students at higher risk of later grade retention and dropout. We find some evidence of positive classroom spillovers. Leveraging existing school inputs to implement a light-touch, cost-effective information intervention can improve education outcomes in lower-income settings. [Funding for this report was provided by The Abdul Latif Jameel Poverty Action Lab and a Chilean FONIDE grant.]

Proponents of school choice argue that it improves educational outcomes by allowing parents to self-select into schools that are most effective for their children. Contrary to these arguments, empirical evidence suggests that parents may not incorporate school effectiveness or match quality when choosing schools. The findings potentially impugn proponents' effectiveness arguments of choice-based assignment. We develop novel solutions that restore effectiveness by maximizing match quality subject to stability constraints. Maximization algorithms are provided for both small and large school districts. Simulations reveal substantial match quality gains from our solutions compared to the celebrated Deferred Acceptance mechanism with a random tie-breaker. Our methodology can be used to optimize for other policy objectives in school choice or other priority-based matching problems.

The aims of this working paper were to investigate whether taking part in out of school activities during primary school is linked with end of primary school attainment and social, emotional and behavioural outcomes, for all children and specifically for children from economically disadvantaged backgrounds. The analysis is based on the Millennium Cohort Study (MCS) survey data linked to administrative data holding the cohort children's Key Stage 1 (KS1) and Key Stage 2 (KS2) attainment scores. In addition to looking at KS2 attainment (total point score, English and maths) we also investigated social, emotional and behavioural outcomes using the Strengths and Difficulties Questionnaire (SDQ) total difficulties and prosocial skills scores. The results presented in this paper are based on logistic and multiple linear regression analysis. Results showed that sports clubs and "other" (unspecified) club participation was positively associated with attainment outcomes at age 11, when controlling for prior attainment. Participating in organised sports or physical activity was also positively linked to social, emotional and behavioural outcomes. Among disadvantaged children, after school club emerged as the only organised activity linked to child outcomes; participation was linked to both higher KS2 attainment and prosocial skills. The implications of these findings for further research, policy and practice are discussed. Appended to the report are: (1) KS2 Attainment regression tables; and (2) SDQ outcomes regression tables. A bibliography is included.

Results of the second year of the New Jersey Minimum Basic Skills Tests in reading and mathematics, mandated by the New Jersey Public School Education Act of 1975, are summarized. Approximately 397,000 students in grades three, six, nine, and eleven were tested. Results of the tests showed that a smaller percentage of sixth-grade students met statewide standards than any other tested grade, confirming previous state and national findings of decreased performance in the intermediate grades. Other findings showed relationships between student achievement and socioeconomic status, type of community in which the school district is located, and region of the state. Tables summarizing results are included, along with appendices outlining required minimum competencies for each grade, and the procedures used to determine cutting scores. (MH)

A staff development needs assessment survey was conducted by the Adult Literacy Resource Institute, in cooperation with other Boston-area programs providing basic education services for adults, to gather information on what users and potential users of the institute wanted in the areas of inservice teacher education, training, and staff development activities for the coming years. Questionnaires were sent individually to 89 staff people of programs within the Boston Adult Literacy Initiative, and to people at 67 other programs (with an unknown number of total staff) doing basic education for adults. These recipients, in turn, were asked to distribute copies of the questionnaire to the staff at their programs. A total of l89 questionnaires were returned, 30 percent from initiative program questionnaires were returned, 30 percent from initiative program staff and 70 percent from other program staff. The rank order of categories of topic areas for inservice training chosen by the respondents were: (1) general basic education for adults (47%); (2) program administration (44%); (3) counseling (36%); (4) teaching English as a Second Language (33%); (5) teaching reading (32%); (6) teaching writing (3l%); (7) teaching in a credential program (20%); (8) teaching math (l6%); and (9) others (6%). (NLL)

We investigate short and long-term effects of early childhood education using variation created by a unique policy experiment in British Columbia, Canada. Our findings imply starting Kindergarten one year late substantially reduces the probability of repeating the third grade, and meaningfully increases in tenth grade math and reading scores. Effects are highest for low income students and males. Estimates suggest that entering kindergarten early may have a detrimental effect on future outcomes.

We present two types of validity evidence to support inferences and decisions about use of easyCBMs in relation to state testing programs. The first type involves the use of Benchmarks in reading to use in making predictions of performance on the Smarter Balanced (SB) test. These predictions can be made both well in advance (several months) or nearly concurrent with administration of the SB test. The second type of validity evidence supports the use of Benchmarks in screening students for risk of problems in learning to read. With data from two states and four school districts, we analyzed well over 8,000 anonymized records, with students reflecting a broad range of demographics. In the first section of the report, we present both the grade level demographics of this sample and their performances on the following easyCBMs in the Fall, Winter, and Spring: Proficient Reading, Vocabulary, Oral (Passage) Reading Fluency, and Math. Criterion validity data (both predictive and concurrent) are first presented for grade level easyCBM raw scores for each measure correlated with the SB test. However, because different CBMs were used within and across grades, we also computed a composite that reflected a standard score (with z-scores computed for each grade level in reading and math). This score transformation allowed both correlations and regression analyses to be presented between easyCBMs and SB using this composite. In the second section of the paper, we present the results of a classification analysis and report a range of statistics: cut points for the 20th percentile rank (PR), classification data (true and false positives and negatives, area under the curve (with 95% confidence intervals for the lower and upper bounds), base rates, overall classification rates, sensitivity and specificity, false positive and negative rates, and finally positive and negative predictive power. The results from these classification analyses support the use easyCBMs as screeners with high AUCs, high values of sensitivity and specificity, and across the board high values of positive predictive power. [This technical report was supported in part by Riverside Insight.]

Ascertaining young children's attitudes towards mathematics has its challenges. Methodologically, limitations exist regarding the type of research techniques that can be employed. The use of children's drawings as a data source has both methodological affordances and issues. The study was conducted with 106 children in Years 2 and 3 from three South Australian primary schools. This paper identifies some of the methodological affordances and issues of using children's drawings to ascertain and describe their attitudes towards mathematics. [This paper is the third in a symposium of three papers. For the first paper, "Drawings Reveal Young Students' Multiplicative Visualisation," see ED616196. For the second paper, "Investigating Students' Drawings as a Representational Mode of Mathematical Fluency," see ED616197.]

The deferred acceptance algorithm proposed by Gale and Shapley (1962) has had a profound influence on market design, both directly, by being adapted into practical matching mechanisms, and, indirectly, by raising new theoretical questions. Deferred acceptance algorithms are at the basis of a number of labor market clearinghouses around the world, and have recently been implemented in school choice systems in Boston and New York City. In addition, the study of markets that have failed in ways that can be fixed with centralized mechanisms has led to a deeper understanding of some of the tasks a marketplace needs to accomplish to perform well. In particular, marketplaces work well when they provide thickness to the market, help it deal with the congestion that thickness can bring, and make it safe for participants to act effectively on their preferences. Centralized clearinghouses organized around the deferred acceptance algorithm can have these properties, and this has sometimes allowed failed markets to be reorganized.