Search

There is a growing interest in ensuring that students are prepared for college and a career when they finish high school. Research identified several indicators that are associated with college preparedness including attendance, grade point average (GPA), and test scores. However, it is difficult to use these indicators to determine the preparedness level of the nation's students in general. This study used results from National Assessment of Educational Progress (NAEP) mathematics assessment, the "gold standard" assessing what students know and can do in grade 12 to establish college preparedness benchmarks. This study linked ninth-grade student background data and school-reported high school and postsecondary transcript data from the National High School Longitudinal Study of 2009 (HSLS:09) to student item responses on the 2013 National Assessment of Educational Progress (NAEP) mathematics assessment to examine the relationship between college preparedness and NAEP mathematics achievement. NAEP scale scores imputed for all HSLS:09 participants via marginal maximum likelihood regression analyses were used to predict college preparedness. This study extended the earlier investigations of college preparedness based on NAEP grade 12 mathematics data from the prediction of college enrollment alone to the prediction of additional postsecondary outcomes including remedial course-taking and first-year college GPA based on postsecondary transcript data. Defining college preparedness as enrolling in a 2- or 4-year college without remedial course-taking and acquiring a first-year college GPA of 2.7 or higher as did National Assessment Governing Board, this study showed that the probability of college preparedness increased from 26 percent for students performing at the NAEP Basic level to 58 percent at the NAEP Proficient level and 87 percent at the NAEP Advanced level. The probabilities at preparedness increased when the definition of college preparedness omitted remedial course-taking. Although similar patterns were observed for various student groups including race/ethnicity, gender, and socioeconomic status, there were some differences in results across groups. Since the report shows the probability of college preparedness (under various definitions) at each NAEP achievement level, it is thereby also providing important validity information for these NAEP achievement levels.

This paper summarises the exploratory quantitative analysis undertaken to investigate how vocational education and training (VET) students cluster and segment in the Australian VET market. This analysis is outlined in three sections. The first section focuses on 'clustering' as a technique for grouping data and the three clustering algorithms used. These are then discussed in more detail to provide some insights into how they operate. Their specific data requirements, along with their strengths and weaknesses, are also considered. In next section the outputs of the clustering approaches are considered. The resultant clusters are examined to better understand them, and meaningfully label and group them into segments. The final section returns to the raw data. This step was necessary to further explore (in this case, only some of) the identified market segments. Three key market segments are explored: students in targeted English programs; students in social inclusion programs; and migrant students.

Science, technology, engineering, and mathematics (STEM) professional societies (ProS) are uniquely positioned to foster national-level diversity, equity, and inclusion (DEI) reform. ProS serve broad memberships, define disciplinary norms and culture, and inform accrediting bodies, thus providing "excellent leverage with which to design and promote change" (National Academy of Sciences et al., 2005). ProS could be instrumental in achieving the DEI culture reform necessary to optimize engagement of all STEM talent by leveraging disciplinary excellence resulting from diverse teams. Inclusive STEM culture reform requires that underlying "mental models'' be examined (Kania et al., 2018). The "Inclusive Professional Framework for Societies (IPF:Societies)" can help ProS change leaders (i.e., "boundary spanners'") and organizations identify and address mental models hindering DEI reform. "IPF:Societies" uses four "I's"--Identity awareness and Intercultural mindfulness (i.e., equity mindset) on which Inclusive relationships and Influential DEI actions are scaffolded. We discuss how "IPF:Societies" complements existing DEI tools. We explain how "IPF:Societies" can be applied to existing ProS policy and practice associated with common ProS functions (e.g., leadership, membership, conferences, awards, and professional development). Next steps are to pilot "IPF:Societies" with a cohort of STEM ProS. Ultimately, "IPF:Societies" has potential to promote more efficient, effective, and lasting DEI organizational transformation, and contribute to inclusive STEM disciplinary excellence.

Among community college leaders and others interested in reforms to improve student success, there is growing interest in adopting machine learning (ML) techniques to predict credential completion. However, ML algorithms are often complex and are not readily accessible to practitioners for whom a simpler set of near-term measures may serve as sufficient predictors. This study compares the out-of-sample predictive power of early momentum metrics (EMMs)--13 near-term success measures suggested by the literature--with that of metrics from ML-based models that employ approximately 500 predictors for community college credential completion. Using transcript data from approximately 50,000 students at more than 30 community colleges in two states, I find that the EMMs that were modeled by logistic regression accurately predict completion for approximately 80% of students. This classification performance is comparable to that of the ML-based models. The EMMs even outperform the ML-based models in probability estimation. These findings suggest that EMMs are useful predictors for credential completion and that the marginal gain from using an ML-based model over EMMs is small for credential completion prediction when additional predictors do not have strong rationales to be included in an ML-based model, no matter how large the number of those predictors may be.

School value-added models are increasingly used to measure schools' contributions to student success. At the same time, policymakers and researchers agree that schools should support students' socialemotional learning (SEL) as well as academic development. Yet, the evidence regarding whether schools can influence SEL and whether statistical growth models can appropriately measure this influence is limited. Recent work shows meaningful differences across schools in changes in SEL scores by grade (Loeb, Christian, Hough, Meyer, Rice, & West, 2019), but whether these differences represent the effects of schools is still unclear. The current paper builds upon this earlier work by examining the stability of the estimated school-by-grade effects on SEL across two years, using a large-scale SEL survey administered in California's CORE districts. We find that correlations among school effects in the same grades across different years are positive, but they are lower than those for math and English Language Arts (ELA). Schools in the top or the bottom of the school effect distribution are more persistent in their impacts across years than those in the middle of the distribution. Overall, the results provide evidence that these school effects measure real contributions to SEL. However, the low stability of effects from one year to the next draw into question whether including these school value-added measures of self-reported SEL in school performance frameworks and systems would be beneficial.

Principals are widely seen as a key influence on the educational environment of schools, and nearly all principals have experience as teachers. Yet there is no evidence on whether we can predict the effectiveness of principals (as measured by their value added) based on their value added as teachers, an issue we explore using administrative data from Washington. Several descriptive features of the principal labor market stand out. First, teachers who become principals tend to have higher levels of educational attainment while teaching and are less likely to be female, but we find no significant differences in licensure test scores between those teachers who become principals and those we do not observe in the principalship. Second, principal labor markets appear to be quite localized: about 50 percent of principals previously taught in the same district in which they assumed a principalship. We find positive correlations between teacher and principal value added in reading (ELA) and similarly sized but less precise estimates in math. Teachers who become principals have slightly higher teacher value added, but the difference between the two groups is not statistically significant, suggesting that principals are not systematically selected based on their prior effectiveness when serving as a classroom teacher.

We present two types of validity evidence to support inferences and decisions about use of easyCBMs in relation to state testing programs. The first type involves the use of Benchmarks in reading to use in making predictions of performance on the Smarter Balanced (SB) test. These predictions can be made both well in advance (several months) or nearly concurrent with administration of the SB test. The second type of validity evidence supports the use of Benchmarks in screening students for risk of problems in learning to read. With data from two states and four school districts, we analyzed well over 8,000 anonymized records, with students reflecting a broad range of demographics. In the first section of the report, we present both the grade level demographics of this sample and their performances on the following easyCBMs in the Fall, Winter, and Spring: Proficient Reading, Vocabulary, Oral (Passage) Reading Fluency, and Math. Criterion validity data (both predictive and concurrent) are first presented for grade level easyCBM raw scores for each measure correlated with the SB test. However, because different CBMs were used within and across grades, we also computed a composite that reflected a standard score (with z-scores computed for each grade level in reading and math). This score transformation allowed both correlations and regression analyses to be presented between easyCBMs and SB using this composite. In the second section of the paper, we present the results of a classification analysis and report a range of statistics: cut points for the 20th percentile rank (PR), classification data (true and false positives and negatives, area under the curve (with 95% confidence intervals for the lower and upper bounds), base rates, overall classification rates, sensitivity and specificity, false positive and negative rates, and finally positive and negative predictive power. The results from these classification analyses support the use easyCBMs as screeners with high AUCs, high values of sensitivity and specificity, and across the board high values of positive predictive power. [This technical report was supported in part by Riverside Insight.]

In the first systematic study of gender achievement gaps in U.S. school districts, we estimate male-female test score gaps in math and English Language Arts (ELA) for nearly 10,000 school districts in the U.S. We use state accountability test data from third through eighth grade students in the 2008-09 through 2014-15 school years. The average school district in our sample has no gender achievement gap in math, but a gap of roughly 0.23 standard deviations in ELA that favors girls. Both math and ELA gender achievement gaps vary among school districts and are positively correlated -- some districts have more male-favoring gaps and some more female-favoring gaps. We find that math gaps tend to favor males more in socioeconomically advantaged school districts and in districts with larger gender disparities in adult socioeconomic status. These two variables explain about one fifth of the variation in the math gaps. However, we find little or no association between the ELA gender gap and either socioeconomic variable, and we explain virtually none of the geographic variation in ELA gaps.

Nearly two dozen states now administer online exams. These tests have real consequences: their results feed into accountability systems, which have been used for more than a decade to hold schools and districts accountable for their students' learning. We examine the rollout of computer-based testing in Massachusetts over 2 years to investigate test mode effects. Crucial to the study design is the fact that the state administers the same exam (PARCC) in online and offline formats each year during the transitional period. We find mode effects of about 0.10 standard deviations in math and 0.25 standard deviations in English language arts (ELA). Our preferred estimates of the effects of online testing in the second year of administration suggest that mode effects for second-time test takers were about one third as large as the first year in math and about half as large in ELA. There is little evidence of systematic variation in mode effects by student demographic groups, although on ELA tests they are larger for students scoring at the bottom of the achievement distribution.

The Common Core State Standards for Mathematics (CCSS-M) set rigorous standards for each of grades 6, 7 and 8. Strategic Education Research Partnership (SERP) has been working with two school districts, Oakland Unified School District and San Francisco Unified School District, to evaluate extant policies and practices and formulate new policies based on the CCSS-M shifts that promise to work better for students. This work has culminated in school board adoptions of new policies in both districts. This report focuses on the problems, solutions and designs that that have been found in the CCSS-M and can be of immediate use to other districts and states.

Understanding the extent engagement and math attitudes predict performance in statistics courses could inform educational interventions in this subject area, which has growing demand. We examined direct and indirect associations between course engagement-related constructs, math attitudes, and learning outcomes. Confirmatory factor analysis was used to validate scores from measures of these constructs with a sample of high school students enrolled in Advanced Placement (AP) Statistics (N = 720, Mean age = 16.8 years, SD = 0.82). Structural equation models were fitted to the data to examine relations between these constructs on a subsample (N = 220). A greater proportion of variation was explained in a high-stakes learning outcome (R[superscript 2] = 0.54) than a low-stakes learning outcome (R[superscript 2] = 0.24). We found some evidence of indirect effects of academic procrastination and course engagement on the learning outcome by way of math attitudes. The findings shed light on opportunities for intervention on academic maladaptive behaviors, such as procrastination, which could lessen negative effects on math attitudes and learning. These findings highlight the importance of testing stakes when examining associations between engagement, math attitudes, and learning, particularly in the context of high school statistics, a growing and yet understudied STEM learning context. [This paper was published in "Journal for STEM Education Research."]

Spatial thinking permeates much of our lives and is an asset when solving problems involving well-structured visual information or imagining solutions in physical or digital space. However, an estimated three million US school children have spatial talents that go unrecognized because of the tools commonly used for identification of academic talent. For decades, educational and psychological research has explored the range of spatial thinking skills that are demanded by many career fields, including science, engineering, and mathematics. Spatial thinking has been found to be particularly important to early mathematical thinking. In this article, we explore what spatial thinking entails, where it is important in the curriculum, and how we can begin to develop spatial literacy and identify spatial talents in our K-12 classrooms. [This paper was published in "Gifted Child Today" v47 n3 p170-177 2024.]

Ascertaining young children's attitudes towards mathematics has its challenges. Methodologically, limitations exist regarding the type of research techniques that can be employed. The use of children's drawings as a data source has both methodological affordances and issues. The study was conducted with 106 children in Years 2 and 3 from three South Australian primary schools. This paper identifies some of the methodological affordances and issues of using children's drawings to ascertain and describe their attitudes towards mathematics. [This paper is the third in a symposium of three papers. For the first paper, "Drawings Reveal Young Students' Multiplicative Visualisation," see ED616196. For the second paper, "Investigating Students' Drawings as a Representational Mode of Mathematical Fluency," see ED616197.]

This report investigates enrollment patterns and academic outcomes in online, hybrid, and face-to-face courses among students who enrolled in Washington State community and technical colleges in the fall of 2004. Students were tracked for nearly five years, until the spring of 2009. Results were similar to those found in a parallel study in Virginia (Jaggars & Xu, 2010). Students who were employed for more hours and students who had demographic characteristics associated with stronger academic preparation were more likely to enroll in online courses; however, students who enrolled in hybrid courses were quite similar to those who enrolled in a purely face-to-face curriculum. After controlling for student characteristics using multilevel regression techniques, results indicated that students were more likely to fail or withdraw from online courses than from face-to-face courses. In addition, students who took online coursework in early terms were slightly but significantly less likely to return to school in subsequent terms, and students who took a higher proportion of credits online were slightly but significantly less likely to attain an educational award or transfer to a four-year institution. In contrast, students were equally likely to complete a hybrid course as to complete a face-to-face course. Additional analyses with a new cohort of students entering in 2008 showed short-term results consistent with those of the 2004 cohort. Given the importance of online learning in terms of student convenience and institutional flexibility, current system supports for online learning should be bolstered and strengthened in order to improve completion rates among online learners. Specific recommendations are discussed in the report's conclusion. (Contains 9 tables, 4 figures and 23 footnotes.)

This paper presents part of the results of a completed study entitled "A Longitudinal Study of Minority Ph.D.s from 1980-1990: Progress and Outcomes in Science and Engineering at the University of California during Graduate School and Professional Life." It focuses particularly on the graduate school experience and degree of preparation for the professoriate of African American doctoral students in the sciences and engineering, and presents the results of a survey of 33 African American STEM Ph.D.s from the University of California earned between 1980-1990. Relationships with thesis advisors and principal investigators are evaluated by the study participants in fifteen specific areas from highly-ranked intellectual development to low-ranked training in grant writing. Deficits in training and socialization are discussed along with the tension between being both an African American and a graduate student. Career choices and outcomes are presented. These findings, in conjunction with current analyses of graduate education in STEM, suggest ways in which graduate training for all could be improved. (Contains 1 table.) [Financial support for this project was provided by the U.C. Industry-University Cooperative Research Program; and the U.C. Berkeley Graduate Division.]

Over the past few years, several states have begun to explore or pilot different through-year assessments to serve as replacements to the traditional end-of-year summative assessments that are currently the predominant source of information used by states to meet federal accountability requirements. While there are several different assessment models that could be considered as part of a through-year design, the purpose of this paper is to examine what it might look like to administer the end-of-year computer adaptive test (CAT) in a through-year model that more closely resembles the current Smarter Balanced interim assessment blocks (IABs), with a specific emphasis on content considerations. It is guided by the following two questions: (1) To what degree do some of the most widely used mathematics and English language arts/literacy curricula support a reasonable structure for clustering standards for a through-year model? and (2) To what degree can the existing Smarter Balanced interim assessment system, with some adaptations, be used to support the design and development of a high-quality through-year assessment model?

This paper reports on the Macromedia Accessibility Project (MAP), a collaborative effort developed by Macromedia, Inc., The League for Innovation in the Community College, and Pima Community College, Arizona, in order to explore the questions and challenges of implementing accessibility standards in online developmental math courses in community colleges. The purpose of the project, as a two-part process, was to evaluate the capacity to create online mathematical content in an accessible format, and to identify the differences between updating content and design differences for Web-based courses versus creating new content and design features for Web-based delivery. Project team members were charged to audit and upgrade one existing online math course and create one new online math course. As a summer project commencing June 15, 2002 and ending September 15, 2002, the 3-month timeline called for inclusion of training sessions, groups meetings, and follow-up communications. Comments from MAP team developers were summarized for review. This paper describes the two different processes teams went through in the course revisions and the new course development. The MAP team members completed all project goals and objectives as designated, yet noted that their work barely scratched the surface. Includes 5 figures and recommendations for further research. (NB)

Tracking is widespread in U.S. education. In post-secondary education alone, at least 71% of colleges use a test to track students. However, there are concerns that the most frequently used college placement exams lack validity and reliability, and unnecessarily place students from under-represented groups into remedial courses. While recent research has shown that tracking can have positive effects on student learning, inaccurate placement has consequences: students face misaligned curricula and must pay tuition for remedial courses that do not bear credits toward graduation. We develop an alternative system to place students that uses predictive analytics to combine multiple measures into a placement instrument. Compared to colleges' existing placement tests, the algorithm is more predictive of future performance. We then conduct an experiment across seven colleges to evaluate the algorithm's effects on students. Placement rates into college-level courses increased substantially without reducing pass rates. Adjusting for multiple testing, algorithmic placement generally, though not always, narrowed gaps in college placement rates and remedial course taking across demographic groups. A detailed cost analysis shows that the algorithmic placement system is socially efficient: it saves costs for students while increasing college credits earned, which more than offsets increased costs for colleges. Costs could be reduced with improved data digitization, as opposed to entering data by hand.

This paper is a report of a study that investigated whether increased private-school competition results in enhanced performance of public schools. Data for the study came from a pooled data set from Georgia Department of Education documents and directories, which included socioeconomic data on students and test scores from Georgia school systems between 1980 and 1990 for 3rd-grade and 10th-grade reading and math tests. The data allowed researchers to address directly the problem of endogeneity. Following an introduction, the report presents a framework of how private competition can affect public-school performance and investigates the conditions for a beneficial competitive effect. The next section explains the empirical models and details the data used in the analysis. The next section presents the empirical results. The empirical results do not lend support to the hypothesis that private-school competition improves public-school performance, as measured by student-exam results. Supplementing the text are five tables that detail test scores and socioeconomic data. (Contains 23 references.) (WFA)

There are a number of issues to consider in choosing tests, including alignment of tests and standards, the integration of tests with curriculum and instruction, the quality of the tests, and a clear definition of the purpose of the test. To address the various needs reflected by these issues, PLATO Learning, Inc., offers two curriculum-wide testing systems: NetSchools Orion GATE and PLATO (registered) LINK. For practice in preparation for high-stakes tests, PLATO Learning offers the Simulated Tests in mathematics, reading, and writing. To support needs for placement, progress control, and cumulative testing when using PLATO (registered) courseware, PLATO Learning offers the FASTRACK and Skills Inventory systems, module mastery tests, and course-level assessments. Each of these systems has different characteristics and is designed to serve different needs. Choosing among them involves answering 12 key questions about testing needs, which are provided in this document. (Author/SLD)

Word-problem features such as text complexity, charts and graphs, position of the unknown, calculation complexity, irrelevant information, and schemas impact word-problem performance. We compared the word-problem performance of typically achieving (TA) students and students with mathematics difficulty (MD). First, we measured the word-problem performance of all students for schemas and position of the unknown. Then, the performance of students with MD for schemas, position of the unknown, irrelevant information, and charts or graphs. Across schemas, while TA students outperformed students with MD, all students typically scored higher on Change and Difference problems than Total problems. For position of the unknown, students often scored highest on problems with the final position unknown. Students with MD also demonstrated higher scores on problems with irrelevant information than charts and graphs. Although patterns emerged, not all problems followed the same trends, suggesting further research should investigate the impact of word problem features on word-problem accuracy. [This paper was published in "Learning Disabilities Research and Practice" v37 n1 p37-50 2022 (EJ1326906).]

This report describes the results from an exploratory project conducted for the National Center for Education Statistics. The purpose of the project was to develop and field test questionnaire items and related methods designed to capture information about the instructional processes used nationally in 8th- to 12th-grade mathematics classrooms. Data from classroom observers, short logs of daily classroom practice, and a teacher questionnaire were used to assess the quality of the items. A survey questionnaire, field tested with 100 teachers, was completed by approximately 300 mathematics teachers, and a case study was conducted with 41 teachers in similar settings with comparable responsibilities. Results of these studies were used to make 10 recommendations to improve the field test validation process and the survey items. Thirteen appendixes contain supplemental information about the study and the test items. Appendix M contains a summary, in chart form, of recommended changes to the field test items. (Contains 21 references.) (SLD)

Grade retention and early dropout are two of the biggest challenges facing education systems in middle-income countries today, representing waste in school resources. We investigate whether reducing parent-school information gaps can improve outcomes that are early-warning signals for grade retention and dropout. We conducted an experiment in low-income schools in Chile to test the effects and behavioral changes triggered by a program that sends attendance, grade, and classroom behavior information to parents via weekly and monthly text messages. Our 18-month intervention raised average math GPA by 0.09 of a standard deviation and increased the share of students satisfying attendance requirements for grade promotion by 4.5 percentage points. Treatment effects were larger for students at higher risk of later grade retention and dropout. We find some evidence of positive classroom spillovers. Leveraging existing school inputs to implement a light-touch, cost-effective information intervention can improve education outcomes in lower-income settings. [Funding for this report was provided by The Abdul Latif Jameel Poverty Action Lab and a Chilean FONIDE grant.]

Factor analysis is often used to model scales created to measure latent constructs, and internal structure validity evidence is commonly assessed with indices like SRMR, RMSEA, and CFI. These indices are essentially effect size measures and definitive benchmarks regarding which values connote reasonable fit have been elusive. Simulations from the 1990s suggesting possible benchmark values are among the most highly cited methodological papers across any discipline. However, simulations have suggested that fixed benchmarks do not generalize well -- fit indices are systematically impacted by characteristics like the number of items and the magnitude of the loadings, so fixed benchmarks can confound misfit with model characteristics. Alternative frameworks for creating customized, model-specific benchmarks have recently been proposed to circumvent these issues but they have not been systematically evaluated. Motivated by two empirical applications where different methods yield inconsistent conclusions, two simulation studies are performed to assess the ability of three different approaches to correctly classify models that are correct or misspecified across different conditions. Results show that dynamic fit indices and equivalence testing both improved upon the traditional Hu & Bentler benchmarks and dynamic fit indices appeared to be least confounded with model characteristics in the conditions studied. [This paper was published in "Multivariate Behavioral Research" v58 n1 p195-219 2023.]

Proponents of school choice argue that it improves educational outcomes by allowing parents to self-select into schools that are most effective for their children. Contrary to these arguments, empirical evidence suggests that parents may not incorporate school effectiveness or match quality when choosing schools. The findings potentially impugn proponents' effectiveness arguments of choice-based assignment. We develop novel solutions that restore effectiveness by maximizing match quality subject to stability constraints. Maximization algorithms are provided for both small and large school districts. Simulations reveal substantial match quality gains from our solutions compared to the celebrated Deferred Acceptance mechanism with a random tie-breaker. Our methodology can be used to optimize for other policy objectives in school choice or other priority-based matching problems.

In response to the COVID-19 pandemic, the Houston Independent School District (HISD) moved to remote learning, which included distance learning, independent study, live online meetings, and completing assignments online or paper-based packets mailed to the home beginning on March 30, 2020. The last nine weeks of the 2019-2020 school year were completed remotely. HISD extended remote learning through the 2020 summer school session and through the first six-week grading cycle of the 2020-2021 school year for all students. The 2020-2021 school year began remotely on September 8, 2020. The option for face-to-face instruction began on October 19, 2020, and parents had the option to choose remote or face-to-face instruction for the fall semester or the entire school year. It is important to understand how students enrolled in HISD performed academically for the second half of the 2019-2020 school year and the 2020-2021 school year, both as compared to the 2018-2019 school year (the most recent pre-pandemic school year) and as compared across different student groups within the district. This report analyzes the grades earned by students in the second semesters of the 2018-2019, 2019-2020, and 2020-2021 school years by grade level category, instructional mode (remote and face-to-face), race/ethnicity, student gender, and special populations including economically disadvantaged students, English learners, gifted/talented, special education, and homeless students. The report further disaggregates second semester grades by high school grade level (9th, 10th, 11th, and 12th) to ensure students are on track to fulfill high school graduation requirements.

The aims of this working paper were to investigate whether taking part in out of school activities during primary school is linked with end of primary school attainment and social, emotional and behavioural outcomes, for all children and specifically for children from economically disadvantaged backgrounds. The analysis is based on the Millennium Cohort Study (MCS) survey data linked to administrative data holding the cohort children's Key Stage 1 (KS1) and Key Stage 2 (KS2) attainment scores. In addition to looking at KS2 attainment (total point score, English and maths) we also investigated social, emotional and behavioural outcomes using the Strengths and Difficulties Questionnaire (SDQ) total difficulties and prosocial skills scores. The results presented in this paper are based on logistic and multiple linear regression analysis. Results showed that sports clubs and "other" (unspecified) club participation was positively associated with attainment outcomes at age 11, when controlling for prior attainment. Participating in organised sports or physical activity was also positively linked to social, emotional and behavioural outcomes. Among disadvantaged children, after school club emerged as the only organised activity linked to child outcomes; participation was linked to both higher KS2 attainment and prosocial skills. The implications of these findings for further research, policy and practice are discussed. Appended to the report are: (1) KS2 Attainment regression tables; and (2) SDQ outcomes regression tables. A bibliography is included.

In response to the COVID-19 pandemic, the Houston Independent School District (HISD) moved to remote learning, which included distance learning, independent study, live online meetings, and completing assignments online or paper-based packets mailed to the home beginning on March 30, 2020. HISD extended remote learning through the end of the 2019-2020 school year, through the 2020 summer school session, and through the first six week grading cycle of the 2020-2021 school year. All students began the school year remotely on September 8, 2020. The option for face-to-face instruction began on October 19, 2020, and parents had the option to choose remote or face-to-face instruction for the fall semester or the entire school year. Students are living through the most disruptive school year of their lives because of the COVID-19 pandemic. They are required to learn at home online or wear masks in classrooms without the benefit of their usual social, sports, and artistic outlets. It is important to understand how students enrolled in HISD are performing academically this school year, both as compared to the prior school year and as compared to different student groups within the district. This report analyzes the grades earned by students in the first semesters of the 2019-2020 and the 2020-2021 school years by grade level, instructional mode (remote and face-to-face), race/ethnicity, and special populations including English learners, gifted/talented, special education, and economically disadvantaged students.

In response to the COVID-19 pandemic, the Houston Independent School District (HISD) moved to remote learning, which included distance learning, independent study, live online meetings, and completing assignments online or paper-based packets mailed to the home beginning on March 30, 2020. HISD extended remote learning through the end of the 2019-2020 school year, through the 2020 summer school session, and through the first six week grading cycle of the 2020-2021 school year. All students began the school year remotely on September 8, 2020. The option for face-to-face instruction began on October 19, 2020, and parents had the option to choose remote or face-to-face instruction for the fall semester or the entire school year. Students are living through the most disruptive school year of their lives because of the COVID-19 pandemic. They are required to learn at home online or wear masks in classrooms without the benefit of their usual social, sports, and artistic outlets. It is important to understand how students enrolled in HISD are performing academically this school year, both as compared to the prior school year and as compared to different student groups within the district. This report analyzes the grades earned by students in the first semesters of the 2019-2020 and the 2020-2021 school years by grade level, instructional mode (remote and face-to-face), race/ethnicity, and special populations including English learners, gifted/talented, special education, and economically disadvantaged students.

The purpose of this study was to analyze the findings of research studies investigating the role of instructional materials, review issues and recommendations referring to instructional materials in policy papers, and evaluate the decision-making process in relation to the development, diffusion and adoption of EdReports program. Content analysis was used to analyze the subject matter of reports and policy papers. A decision-oriented evaluation model was used to analyze planning, structuring, implementing and recycling decisions occurring in the change process within EdReports.org. The results showed that the wave of research investigating the role of materials has continued with publication of new reports, the release of policy papers, initiation of a study investigating adoption patterns, and commencement of new projects by EdReports.org. The findings showed that researchers are investigating the effects of decision-making on the selection of high-quality, standards-aligned materials, foundations and non-profit organizations are providing recommendations to policymakers to link high-quality, standards-aligned materials with professional learning, and EdReports.org has developed, diffused and adopted a program to provide information about high-quality, standards-aligned materials to the education community.

We investigated the feasibility of using automatic speech recognition (ASR) and natural language processing (NLP) to classify collaborative problem solving (CPS) skills from recorded speech in noisy environments. We analyzed data from 44 dyads of middle and high school students who used videoconferencing to collaboratively solve physics and math problems (35 and 9 dyads in school and lab environments, respectively). Trained coders identified seven cognitive and social CPS skills (e.g., sharing information) in 8,660 utterances. We used a state-of-the-art deep transfer learning approach for NLP, Bidirectional Encoder Representations from Transformers (BERT), with a special input representation enabling the model to analyze adjacent utterances for contextual cues. We achieved a micro-average AUROC score (across seven CPS skills) of 0.80 using ASR transcripts, compared to 0.91 for human transcripts, indicating a decrease in performance attributable to ASR error. We found that the noisy school setting introduced additional ASR error, which reduced model performance (micro-average AUROC of 0.78) compared to the lab (AUROC = 0.83). We discuss implications for real-time CPS assessment and support in schools. [For the full proceedings, see ED615472.]

Digitalization and automation of test administration, score reporting, and feedback provision have the potential to benefit large-scale and formative assessments. Many studies on automated essay scoring (AES) and feedback generation systems were published in the last decade, but few connected AES and feedback generation within a unified framework. Recent advancements in machine learning algorithms enable researchers to develop more models that explore the potential of automated assessments in education. This study makes the following contributions. First, it implements, compares, and contrasts three AES algorithms with word-embedding and deep learning models (CNN, LSTM, and BiLSTM). Second, it proposes a novel automated feedback generation algorithm based on the Constrained Metropolis-Hastings Sampling (CGMH). Third, it builds a classifier to integrate AES and feedback generation into a systematic framework. Results show that: (1) the scoring accuracy of the AES algorithm outperforms that of state-of-the-art models; and (2) the CGMH method generates semantically-related feedback sentences. The findings support the feasibility of an automated system that combines essay scoring with feedback generation. Implications may lead to the development of models that reveal linguistic features, while achieving high scoring accuracy, as well as to the creation of feedback corpora to generate more semantically-related and sentiment-appropriate feedback. [For the full proceedings, see ED615472.]

Traditionally, clustering algorithms focus on partitioning the data into groups of similar instances. The similarity objective, however, is not sufficient in applications where a "fair-representation" of the groups in terms of protected attributes like gender or race, is required for each cluster. Moreover, in many applications, to make the clusters useful for the end-user, a "balanced cardinality" among the clusters is required. Our motivation comes from the education domain where studies indicate that students might learn better in diverse student groups and of course groups of similar cardinality are more practical e.g., for group assignments. To this end, we introduce the "fair-capacitated clustering problem" that partitions the data into clusters of similar instances while ensuring cluster fairness and balancing cluster cardinalities. We propose a two-step solution to the problem: (1) we rely on fairlets to generate minimal sets that satisfy the fair constraint; and (2) we propose two approaches, namely hierarchical clustering and partitioning-based clustering, to obtain the fair-capacitated clustering. Our experiments on three educational datasets show that our approaches deliver well-balanced clusters in terms of both fairness and cardinality while maintaining a good clustering quality. [For the full proceedings, see ED615472.]

Recently Prof. Howard Gardner, an outstanding psychologist in the worldwide accepted the interview from Dr. Weiwen Zhang, and talked about a wide range of MI theory and relevant fields, which mainly involved in its core ideas, current situation and future development, and also involved its application in some current hot issues, which gave us important enlightenment in relevant fields.

Multidimensional Item Response Theory (MIRT) is widely used in assessment and evaluation of educational and psychological tests. It models the individual response patterns by specifying functional relationship between individuals' multiple latent traits and their responses to test items. One major challenge in parameter estimation in MIRT is that the likelihood involves intractable multidimensional integrals due to latent variable structure. Various methods have been proposed that either involve direct numerical approximations to the integrals or Monte Carlo simulations. However, these methods are known to be computationally demanding in high dimensions and rely on sampling data points from a posterior distribution. We propose a new Gaussian Variational EM (GVEM) algorithm which adopts a variational inference to approximate the intractable marginal likelihood by a computationally feasible lower bound. In addition, the proposed algorithm can be applied to assess the dimensionality of the latent traits in an exploratory analysis. Simulation studies are conducted to demonstrate the computational efficiency and estimation precision of the new GVEM algorithm in comparison to the popular alternative Metropolis-Hastings Robbins-Monro (MHRM) algorithm. In addition, theoretical results are also presented to establish the consistency of the estimator from the new GVEM algorithm. [This paper will be published in "British Journal of Mathematical and Statistical Psychology."]

Collaborative problem solving (CPS) is an important 21st-century skill for academic and career success, and as a result, there is increased interest among businesses and educational institutions in the assessment and development of CPS skills. CPS skills are difficult to measure using traditional forms of assessment, and that difficulty has led to the use of computer environments that allow individuals to interact in complex situations and capture all actions throughout the process. In the current paper, we describe the design of a collaborative online mathematics task and explore student perceptions of the task and the extent to which the task elicits CPS skills. Results revealed areas of difficulty for students and showed that the task mostly elicited skills associated with the social dimension of CPS. We describe modifications to the task to provide better opportunities for students to display a broader range of CPS skills. An upcoming larger scale pilot study will explore whether the redesigned task can elicit the less observed CPS skills.

Item factor analysis (IFA), also known as Multidimensional Item Response Theory (MIRT), is a general framework for specifying the functional relationship between a respondent's multiple latent traits and their response to assessment items. The key element in MIRT is the relationship between the items and the latent traits, so-called item factor loading structure. The correct specification of this loading structure is crucial for accurate calibration of item parameters and recovery of individual latent traits. This paper proposes a regularized Gaussian Variational Expectation Maximization (GVEM) algorithm to efficiently infer item factor loading structure directly from data. The main idea is to impose an adaptive L[subscript 1]-type penalty to the variational lower bound of the likelihood to shrink certain loadings to 0. This new algorithm takes advantage of the computational efficiency of GVEM algorithm and is suitable for high-dimensional MIRT applications. Simulation studies show that the proposed method accurately recovers the loading structure and is computationally efficient. The new method is also illustrated using the National Education Longitudinal Study of 1988 (NELS:88) mathematics and science assessment data. [This paper will be published in "Psychometrika."]

Adult numeracy is one of the essential skill sets to navigate through numeric information-rich labor markets in general, and STEM industries in particular. Yet, relatively little is known about how numeracy skills are used in different settings in the USA. This study examined numeracy skill use patterns of STEM and non-STEM workers at work and home. Data were obtained from the 2012/2014/2017 Program for International Assessment of Adult Competencies, USA restricted-use file. Adults who were employed and aged between 25 and 65 years old (n = 5,220) were included in this study. Latent class analysis revealed four numeracy skill use patterns: non-users, non-occupational (i.e., at home) simple numeracy users, ubiquitous numeracy users, and occupational numeracy users. Additional multinomial logistic regression analysis showed that the STEM occupation was associated with a greater likelihood of being ubiquitous users than being non-occupational simple users. Results also showed that numeracy proficiency, socioeconomic statuses (i.e., educational attainment and income), as well as demographic characteristics (i.e., gender and race/ethnicity), were predictive of the numeracy skill use patterns in terms of the level of engagement and settings. Findings from this study inform policies and interventions which promote skill engagement and improvement among workers in the USA. [This paper will be published in "International Journal of Lifelong Education."]

Probabilistic machine learning increasingly informs critical decisions in medicine, economics, politics, and beyond. To aid the development of trust in these decisions, we develop a taxonomy delineating where trust in an analysis can break down: (1) in the translation of real-world goals to goals on a particular set of training data, (2) in the translation of abstract goals on the training data to a concrete mathematical problem, (3) in the use of an algorithm to solve the stated mathematical problem, and (4) in the use of a particular code implementation of the chosen algorithm. We detail how trust can fail at each step and illustrate our taxonomy with two case studies. Finally, we describe a wide variety of methods that can be used to increase trust at each step of our taxonomy. The use of our taxonomy highlights steps where existing research work on trust tends to concentrate and also steps where building trust is particularly challenging. [This paper was published in "Science Advances."]

This paper describes the genesis of YuMi Deadly Maths, a school change process that has been used in over 200 schools to develop mathematics teaching and learning to improve students' employment and life chances. The paper discusses the YuMi Deadly Maths approach to mathematics content and pedagogy, implemented through a process of PD and school change, and looks at the strengths and weaknesses of the process and the challenges it faces.

We consider the problem of off-policy policy selection in reinforcement learning: using historical data generated from running one policy to compare two or more policies. We show that approaches based on importance sampling can be "unfair"--they can select the worse of two policies more often than not. We give two examples where the unfairness of importance sampling could be practically concerning. We then present sufficient conditions to theoretically guarantee fairness and a related notion of safety. Finally, we provide a practical importance sampling-based estimator to help mitigate one of the systematic sources of unfairness resulting from using importance sampling for policy selection. [This paper was published in "Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence" (33rd, Sydney, Australia, August 11-15, 2017).]

This study presents a novel video recommendation system for an algebra virtual learning environment (VLE) that leverages ideas and methods from engagement measurement, item response theory, and reinforcement learning. Following Vygotsky's Zone of Proximal Development (ZPD) theory, but considering low affect and high affect students separately, we developed a system of five categories of video recommendations: (1) Watch new video; (2) Review current topic video with a new tutor; (3) Review segment of current video with current tutor; (4) Review segment of current video with a new tutor; and (5) Watch next video in curriculum sequence. The category of recommendation was determined by student scores on a quiz and a sensor-free engagement detection model. New video recommendations (i.e., category 1) were selected based on a novel reinforcement learning algorithm that takes input from an item response theory model. The recommendation system was evaluated in a large field experiment, both before and after school closures due to the COVID-19 pandemic. The results show evidence of effectiveness of the video recommendation algorithm during the period of normal school operations, but the effect disappears after school closures. Implications for teacher orchestration of technology for normal classroom use and periods of school closure are discussed.

Presented are papers and programs generated as part of a Madison, Wisconsin project to develop a public school educational program for severely handicapped students. Three introductory papers deal with topics such as teacher training, litigation over questions of overinclusion and overexclusion, and what the practitioner needs from the researcher to develop public school programs for low-functioning individuals. Four language and reading skill programs are reported on the following topics: teaching pronouns, developing chart story reading skills, and improving reading comprehension. In the area of math, a developmental math skill sequence and a method for teaching students to solve addition problems are provided. Three papers on development of independent community living skills include an instructional model for developing home living skills, a family-life curriculum, and a method for teaching use of the telephone. Also included are a paper on training parents in child management and one on increasing the speech intensity of a retarded-emotionally disturbed student. (LS)

Computer scientists have reported on "computational thinking," which Aho (2012) defines as "the thought process involved in formulating problems so their solutions can be represented as computational steps and algorithms" (p. 832). We wanted to investigate such thinking in mathematics. To study this, we interviewed five mathematicians about the role of computation in their work, and the notion of "algorithmic thinking" developed from these interviews. We found that mathematicians value this thinking, and in this paper we define algorithmic thinking and present a number of applications and affordances of such thinking. [For the complete proceedings, see ED583608.]

This paper provides a description and rationale for a performance assessment of a teaching practice--leading a classroom discussion (LCD)--included in the ETS® National Observational Teaching Examination (NOTE) assessment series. In this assessment, candidates interact with a small class of virtual students represented by avatars in a computer-based, simulated classroom. The five avatars are enacted by a single simulation specialist who has been trained and certified on the particular task presented, either in elementary English language arts or mathematics. The paper defines and describes the construct of LCD, then provides a review of the research and scholarly literature that supports the importance of this practice for effective teaching, and finally describes how the construct is measured in the NOTE assessment. The following is appended: Leading Classroom Discussion (LCD) Sample Task. [Some of the content in this report is used in the following companion reports: "Eliciting Student Thinking: Definition, Research Support, and Measurement of the 'ETS'® National Observational Teaching Examination (NOTE) Assessment Series. Research Memorandum No. RM-16-06" (ED570608); and "Modeling and Explaining Content: Definition, Research Support, and Measurement of the 'ETS'® National Observational Teaching Examination (NOTE) Assessment Series. Research Memorandum No. RM-16-07" (ED570617).]

Between 2003 and 2013, the proportion of California 8th graders enrolled in Algebra or a more advanced course nearly doubled to 65 percent. In this paper, we consider the organizational processes that accompanied this curricular intensification. Facing a complex set of accountability, institutional, technical/functional, and internal political pressures, California schools responded to the Algebra-for-all effort in diverse ways. While some schools detracked by enrolling all 8th graders in Algebra, others "tracked up," creating more advanced geometry opportunities while increasing algebra enrollments. These responses created a new differentiated course structure that is likely to benefit advantaged students. Consistent with the Effectively Maintained Inequality hypothesis, we find that detracking occurred primarily in disadvantaged schools while "tracking up" occurred primarily in advantaged schools. [This paper was published in the "American Education Research Journal" v53 n4 p1229-1266 Aug 2016 (EJ1111102).]

Statisticians have made great progress in creating methods that reduce our reliance on parametric assumptions. However this explosion in research has resulted in a breadth of inferential strategies that both create opportunities for more reliable inference as well as complicate the choices that an applied researcher has to make and defend. Relatedly, researchers advocating for new methods typically compare their method to at best 2 or 3 other causal inference strategies and test using simulations that may or may not be designed to equally tease out flaws in all the competing methods. The causal inference data analysis challenge, "Is Your SATT Where It's At?", launched as part of the 2016 Atlantic Causal Inference Conference, sought to make progress with respect to both of these issues. The researchers creating the data testing grounds were distinct from the researchers submitting methods whose efficacy would be evaluated. Results from 30 competitors across the two versions of the competition (black box algorithms and do-it-yourself analyses) are presented along with post-hoc analyses that reveal information about the characteristics of causal inference strategies and settings that affect performance. The most consistent conclusion was that methods that flexibly model the response surface perform better overall than methods that fail to do so. Finally new methods are proposed that combine features of several of the top-performing submitted methods. [This paper will be published in "Statistical Science."]

In educational and psychological measurement, researchers and/or practitioners are often interested in examining whether the ability of an examinee is the same over two sets of items. Such problems can arise in measurement of change, detection of cheating on unproctored tests, erasure analysis, detection of item preknowledge etc. Traditional frequentist approaches that are used in such problems include the Wald test, the likelihood ratio test, and the score test (e.g., Fischer, 2003; Finkelman, Weiss, & Kim-Kang, 2010; Glas & Dagohoy, 2007; Guo & Drasgow, 2010; Klauer & Rettig, 1990; Sinharay, 2017). This paper shows that approaches based on higher-order asymptotics (e.g., Barndorff-Nielsen & Cox, 1994; Ghosh, 1994) can also be used to test for the equality of the examinee ability over two sets of items. The modified signed likelihood ratio test (e.g. Barndorff-Nielsen, 1986) and the Lugannani-Rice approximation (Lugannani & Rice, 1980), both of which are based on higher-order asymptotics, are shown to provide some improvement over the traditional frequentist approaches in three simulations. Two real data examples are also provided. [This is an advance online version of an article published in "Psychometrika."]

Building the process of education effectively and qualitatively taking into account educational needs and students' individual characteristics is possible by developing of the special technics of education, in terms of which a student not only will repeat the actions of a teacher but first of all will use different interactive and adaptive educational technologies. The modern computer technologies because of high crucible carrier size and high processing speed allow to enlarge the borders of materials used in an educational process because of highly developed ways of analysis of a dialogue with a user and considerably adopt transfer of knowledge to a concrete learner. The teaching technique of computer testing suggests realization of new approaches to a building of teaching courses by special organization of information block of a discipline in the form of sections and modules, structuring of test materials due to the module division in accordance with information structure of learning material, division of module testing material into the groups of test questions and tasks in keeping with the content and arrangement of groups in order of difficulty of testing materials, appliance of consistent approach allowing to realize several sorts of adaptiveness while testing students' knowledge. We offered the consistent approach, allowing to realize practically several types of adaptiveness while estimating knowledge by computer testing. (1) The changing of strategy of examination depending on the student's answers, legalized by means of special structure of examining algorithm. (2) Testing on assignment, which is divided into the training, realized independently for the estimation of knowledge by a student himself, and control, realized under the supervision of a teacher for the objective estimation of real knowledge, and progress check and final check are important and particular case. (3) Testing on profoundness of the examination -- to the examination level. (4) The testing by volume of the checked material subdivided depending on material volume (the module, the section or several sections (progress check), all discipline (final check). The given possibilities are provided with the organization of the information and testing material for subsections of a course and are considered in the testing algorithms. The offered approach maximally takes into account essential characteristics of computer process of an examination and allows to build an individual path of training students by using various interactive means. [For "NORDSCI International Conference Proceedings: Education and Language Edition (Helsinki, Finland, July 17, 2018). Book 1. Volume 1," see ED603189.]

The goal of the current study is to develop a more complete understanding of the early "home math environment," encompassing both numeracy and non-numeracy aspects of that environment. Parents of preschoolers (n = 63) were surveyed about their support of three components of early mathematics knowledge (i.e., numeracy, spatial, and pattern) as well as parents' math-related beliefs about themselves and their children. Children were administered a broad math knowledge assessment which included a numeracy subscale, and individual measures of spatial and patterning skills in the fall (concurrently). Broad math knowledge was measured again in the spring of the preschool year. Parents indicated providing some support of early math development through numeracy, spatial, and patterning activities, with a stronger emphasis on numeracy than pattern and space. Parents' child-specific ability beliefs were related to their numeracy, pattern, and broad math support, while their parent-specific ability beliefs were related to their spatial support. Parent support was rarely linked to child skills, except that numeracy support related to concurrent numeracy knowledge. Findings suggest that although parents do support a broad range of early math skills at home, parents tend to prioritize supporting early numeracy. Parents' beliefs, especially about their child's academic abilities, may influence components of the early home math environment, but future research is needed to better understand the relations between parent's academic beliefs and the home math environment they create. [This paper will be published in "Early Childhood Research Quarterly."]