5 results on '"Adam Sales"'
Search Results
2. Student Log-Data from a Randomized Evaluation of Educational Technology: A Causal Case Study
- Author
-
John F. Pane and Adam Sales
- Subjects
FOS: Computer and information sciences ,Computer science ,4. Education ,Principal stratification ,education ,05 social sciences ,Educational technology ,050301 education ,Mathematics curriculum ,Statistics - Applications ,Education ,law.invention ,Randomized controlled trial ,law ,Log data ,Causal inference ,0502 economics and business ,ComputingMilieux_COMPUTERSANDEDUCATION ,Mathematics education ,Applications (stat.AP) ,Statistical analysis ,050207 economics ,0503 education ,Causal model - Abstract
Randomized evaluations of educational technology produce log data as a bi-product: highly granular data student and teacher usage. These datasets could shed light on causal mechanisms, effect heterogeneity, or optimal use. However, there are methodological challenges: implementation is not randomized and is only defined for the treatment group, and log datasets have a complex structure. This paper discusses three approaches to help surmount these issues. One approach uses data from the treatment group to estimate the effect of usage on outcomes in an observational study. Another, causal mediation analysis, estimates the role of usage in driving the overall effect. Finally, principal stratification estimates overall effects for groups of students with the same "potential" usage. We analyze hint data from an evaluation of the Cognitive Tutor Algebra I curriculum using these three approaches, with possibly conflicting results: the observational study and mediation analysis suggest that hints reduce posttest scores, while principal stratification finds that treatment effects may be correlated with higher rates of hint requests. We discuss these mixed conclusions and give broader methodological recommendations., Forthcoming, Journal of Research in Educational Effectiveness
- Published
- 2020
3. Review
- Author
-
Adam Sales
- Subjects
Mediation (statistics) ,Computer science ,05 social sciences ,01 natural sciences ,Outcome (game theory) ,0506 political science ,Education ,010104 statistics & probability ,Simple (abstract algebra) ,Causal inference ,050602 political science & public administration ,0101 mathematics ,Set (psychology) ,Social Sciences (miscellaneous) ,Statistical software ,Causal mediation ,Cognitive psychology - Abstract
Causal mediation analysis is the study of mechanisms—variables measured between a treatment and an outcome that partially explain their causal relationship. The past decade has seen an explosion of research in causal mediation analysis, resulting in both conceptual and methodological advancements. However, many of these methods have been out of reach for applied quantitative researchers, due to their complexity and the difficulty of implementing them in standard statistical software distributions. The mediation package in R provides a set of simple commands that execute some of the newer causal mediation methods. This article will summarize some of the recent advances in mediation analysis, critically review the mediation package, and demonstrate, by example, some of its capabilities.
- Published
- 2016
4. The role of mastery learning in an intelligent tutoring system: Principal stratification on a latent variable
- Author
-
Adam Sales and John F. Pane
- Subjects
0301 basic medicine ,Statistics and Probability ,Principal stratification ,educational technology ,01 natural sciences ,Algebra I ,Bayesian ,Intelligent tutoring system ,principal stratification ,010104 statistics & probability ,03 medical and health sciences ,Mathematics education ,0101 mathematics ,TUTOR ,Curriculum ,computer.programming_language ,latent variables ,Educational technology ,Cognitive tutor ,item response theory ,Mastery learning ,030104 developmental biology ,Modeling and Simulation ,Statistics, Probability and Uncertainty ,computer ,Causal inference - Abstract
Students in Algebra I classrooms typically learn at different rates and struggle at different points in the curriculum—a common challenge for math teachers. Cognitive Tutor Algebra I (CTA1), an educational computer program, addresses such student heterogeneity via what they term “mastery learning,” where students progress from one section of the curriculum to the next by demonstrating appropriate “mastery” at each stage. However, when students are unable to master a section’s skills even after trying many problems, they are automatically promoted to the next section anyway. Does promotion without mastery impair the program’s effectiveness? ¶ At least in certain domains, CTA1 was recently shown to improve student learning on average in a randomized effectiveness study. This paper uses student log data from that study in a continuous principal stratification model to estimate the relationship between students’ potential mastery and the CTA1 treatment effect. In contrast to extant principal stratification applications, a student’s propensity to master worked sections here is never directly observed. Consequently we embed an item-response model, which measures students’ potential mastery, within the larger principal stratification model. We find that the tutor may, in fact, be more effective for students who are more frequently promoted (despite unsuccessfully completing sections of the material). However, since these students are distinctive in their educational strength (as well as in other respects), it remains unclear whether this enhanced effectiveness can be directly attributed to aspects of the mastery learning program.
- Published
- 2019
5. Rebar: Reinforcing a Matching Estimator with Predictions from High-Dimensional Covariates
- Author
-
Brian Rowan, Adam Sales, and Ben B. Hansen
- Subjects
FOS: Computer and information sciences ,Matching (statistics) ,Computer science ,Computation ,05 social sciences ,Rebar ,050301 education ,Estimator ,Regression analysis ,01 natural sciences ,Statistics - Applications ,Education ,law.invention ,Methodology (stat.ME) ,010104 statistics & probability ,law ,Causal inference ,Statistics ,Covariate ,Applications (stat.AP) ,Observational study ,0101 mathematics ,0503 education ,Social Sciences (miscellaneous) ,Statistics - Methodology - Abstract
In causal matching designs, some control subjects are often left unmatched, and some covariates are often left unmodeled. This article introduces "rebar," a method using high-dimensional modeling to incorporate these commonly discarded data without sacrificing the integrity of the matching design. After constructing a match, a researcher uses the unmatched control subjects--the remnant--to fit a machine learning model predicting control potential outcomes as a function of the full covariate matrix. The resulting predictions in the matched set are used to adjust the causal estimate to reduce confounding bias. We present theoretical results to justify the method's bias-reducing properties as well as a simulation study that demonstrates them. Additionally, we illustrate the method in an evaluation of a school-level comprehensive educational reform program in Arizona., Published in Journal of Educational and Behavioral Statistics (Currently 12/6/17 "Online First")
- Published
- 2015
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.