Comparison of Methods to Trace Multiple Subskills: Is LR-DBN Best?

A long-standing challenge for knowledge tracing is how to update estimates of multiple subskills that underlie a single observable step. We characterize approaches to this problem by how they model knowledge tracing, fit its parameters, predict performance, and update subskill estimates. Previous methods allocated blame or credit among subskills in various ways based on strong assumptions about their relation to observed performance. LR-DBN relaxes these assumptions by using logistic regression in a Dynamic Bayes Net. LR-DBN significantly outperforms previous methods on data sets from reading and algebra tutors in terms of predictive accuracy on unseen data, cutting the error rate by half. An ablation experiment shows that using logistic regression to predict performance helps, but that using it to jointly estimate subskills explains most of this dramatic improvement. An implementation of LR-DBN is now publicly available in the BNT-SM student modeling toolkit. Instruction to use LR-DBN in BNT-SM is appended. (Contains 2 figures, 7 tables, and 2 footnotes.) [For the complete proceedings, "Proceedings of the International Conference on Educational Data Mining (EDM) (5th, Channa, Greece, June 19-21, 2012)," see ED537074.]