Search

This paper describes an initial pilot study of Rimac, a natural-language tutoring system for physics. Rimac uses a student model to guide decisions about "what content to discuss next" during reflective dialogues that are initiated after students solve quantitative physics problems, and "how much support to provide" during these discussions, that is, domain contingent scaffolding and instructional contingent scaffolding, respectively. The pilot study compared an experimental and control version of Rimac. The experimental version uses students' responses to pretest items to initialize the student model and dynamically updates the model based on students' responses to tutor questions during reflective dialogues. It then decides what and how to discuss the next question based on the model predictions. The control version initializes its student model based on students' pretest performance but does not update the model further and assigns students to a fixed line of reasoning level based on the student model predictions. We hypothesized that students who used the experimental version of Rimac would achieve higher learning gains than students who used the control version. Although we did not find a significant difference in learning between conditions, the experimental group took significantly less time to complete the pilot study dialogues than did the control group. That is, the experimental condition led to more efficient learning, for both low and high prior knowledge level learners. We discuss this finding and describe future work to improve the tutor's potential to support student learning.

This paper discusses how a dialogue-based tutoring system makes decisions to proactively scaffold students during conceptual discussions about physics. The tutor uses a student model to predict the likelihood that the student will answer the next question in a dialogue script correctly. Based on these predictions, the tutor will, step by step, choose the granularity at which the next step in the dialogue is discussed. The tutor attempts to pursue the discussion at the highest possible level, with the goal of helping the student achieve mastery, but with the constraint that the questions it asks are within the student's ability to answer when appropriately supported; that is, the tutor aims to stay within its estimate of the student's zone of proximal development for the targeted concepts. The scaffolding provided by the tutor is further adapted by adjusting the way the questions are expressed.

In this paper, we propose a computational approach to model the Zone of Proximal Development (ZPD) using predicted probabilities of correctness and engaging students in reflective dialogue. To that end, we employ a predictive model that uses a linear function of a variety of parameters, including difficulty and student knowledge and we analyze the activity of students who use a natural language tutoring system that presents conceptual reflection questions after students solve high-school physics problems. In order to operationalize our approach, we introduce the concept of the "Grey Area", that is the area of uncertainty in which the student model cannot predict with acceptable accuracy whether a student is able to give a correct answer without support. We further discuss the impact of our approach on student modeling, the limitations of this work and we discuss future work in systematically and rigorously evaluating the approach.

In this paper, we propose a computational approach to modeling the Zone of Proximal Development of students who learn using a natural language tutoring system for physics. We employ a student model that predicts students' performance based on their prior knowledge and their activity when using a dialogue tutor to practice with conceptual, reflection questions about high-school physics. Furthermore, we introduce the concept of the "Grey Area", the area in which the student model cannot predict with acceptable accuracy whether a student has mastered the knowledge components or skills present in a particular step. We envision that our approach will contribute to the way we design learning content for ITSs and the way we author dialogues for natural-language tutoring systems. We further discuss the impact of our approach on student modeling and discuss future work in systematically and rigorously evaluating the approach.

Intelligent tutoring systems (ITS), like human tutors, try to adapt to student's knowledge level so that the instruction is tailored to their needs. One aspect of this adaptation relies on the ability to have an understanding of the student's initial knowledge so as to build on it, avoiding teaching what the student already knows and focusing on the knowledge the student lacks or understands poorly. One way of acquiring this initial student knowledge state is by having the student take a multiple-choice test. However, the overall results commonly provided by multiple-choice tests may not be at the level of granularity needed by the ITS. This paper presents a tool that allows the extraction of fine-grained knowledge from correct and incorrect answers given in multiple-choice tests. Although the tool was developed to be used by ITSs, we argue that it could become a useful instrument for teachers in classroom evaluations.

We explore the effectiveness of a simple algorithm for adaptively deciding whether to further decompose a step in a line of reasoning during tutorial dialogue. We compare two versions of a tutorial dialogue system, Rimac: one that always decomposes a step to its simplest sub-steps and one that adaptively decides to decompose a step based on a student's pre-test assessment. We hypothesize that students using the two versions of Rimac will learn similarly but that students who use the version that adaptively decomposes a step will learn more efficiently. Our initial results suggest support for our hypothesis but the sample size for the experiment is still small and we are continuing to collect more student interactions with the two versions of the system.

Prior research aimed at identifying linguistic features of tutoring that predict learning found interactions between student characteristics (e.g., incoming knowledge level, gender, and affect) and learning. This paper addresses the question: "What do these interactions suggest for developing adaptive natural-language tutoring systems?" We summarize two studies which investigated interactions between gender and learning in a tutoring system for conceptual physics. We argue that student characteristics such as gender are insufficient to guide the development of adaptive tutorial dialogue systems. A more fruitful approach is to consider the underlying factors and cognitive mechanisms that potentially mediate the relationship between student characteristics and learning from particular tutoring tactics.

Although restating part of a student's correct response correlates with learning and various types of restatements have been incorporated into tutorial dialogue systems, this tactic has not been tested in isolation to determine if it causally contributes to learning. When we explored the effect of tutor restatements that support inference on student learning, it did not benefit all students equally. We found that students with lower incoming knowledge tend to benefit more from an increased level of these types of restatement while students with higher incoming knowledge tend to benefit more from a decreased level of such restatements. This finding has implications for tutorial dialogue system design since an inappropriate use of restatements could dampen learning. [This paper was published in: "Proceedings of the SIGDIAL 2015 Conference," Association for Computational Linguistics, 2015, pp. 51-59.]

Tutorial dialogue systems often simulate tactics used by experienced human tutors such as restating students' dialogue input. We investigated whether the amount of tutor restatement that supports student inference interacts with students' incoming knowledge level in predicting how much students learn from a system. We found that students with lower incoming knowledge benefit more from an increased level of these types of restatement while students with higher incoming knowledge benefit more from a decreased level of such restatements.

Although research on human tutoring highlights the importance of a high degree of "interactivity" between the tutor and student, some instructional strategies that could be carried out interactively are implemented didactically in tutoring systems. This is especially true of summarization, a ubiquitous instructional strategy. We investigated summarization during human tutoring in order to determine how to refine decision rules that specify when and how summarization takes place in a tutoring system so that these rules can be made more interactive and adaptive. We describe the informational requirements for carrying out these rules and implications for developing an authoring framework that can provide this information to a dialogue management system.