Search

Mathematics is often taught by explaining an idea, then giving students practice in applying it. Tutoring systems can increase the effectiveness of this method by monitoring the students' practice and giving feedback. However, math can also be taught by having students work collaboratively on problems that lead them to discover the idea. Here, teachers spend the bulk of their time orchestrating collaborations and supporting students in building productively on each other's contributions. Our research question is: Can tutoring technology somehow make teaching-by-eliciting more effective? Using tutoring technology, we developed an intelligent orchestration system named FACT. While students solve problems in small groups, it makes recommendations to the teacher about which groups to visit and what to say. Data from over 50 iterative development trials (study 1) suggest that FACT increased neither the collaboration nor productivity of the students' struggle compared to paper-based classes. However, the data also suggest that when there is just one teacher in the classroom, then only a few of the groups that need a visit can get one. We modified FACT to directly send students the provocative questions that it formerly sent only to teachers. A pilot test (study 2) suggests that this version may increase productive struggle, but increasing collaboration remains an unsolved problem.

Despite their drawback, multiple-choice questions are an enduring feature in instruction because they can be answered more rapidly than open response questions and they are easily scored. However, it can be difficult to generate good incorrect choices (called "distractors"). We designed an algorithm to generate distractors from a semantic network for four types of multiple choice questions in biology. By recruiting 200 participants from Amazon Mechanical Turk, the machine-generated distractors were compared to human-generated distractors in terms of question difficulty, question discrimination and distractor usefulness. The machine-generated and human-generated distractors performed very closely on all the three measures, suggesting that generating distractors from a semantic network for simple multiple choice questions is a viable method.

An algebraic model uses a set of algebra equations to precisely describe a situation. Constructing such models is a fundamental skill required by US standards for both math and science. It is usually taught with algebra word problems. However, many students still lack the skill, even after taking several algebra courses in high school and college. We are developing a short, intensive course in algebraic model construction. The course combines human teaching with a tutoring system. This paper describes the lessons learned during the iterative development process. Starting from an existing theory of model construction, we gradually acquired a completely different view of the skills required as we modified the tutoring system and the instruction. We close by describing encouraging results from a quasi-experimental study.

When a small group of students collaborate, learning gains are often proportional to the amount of co-construction in theirdialogue. Co-construction (also called transactivity or co-explaining) is an observable behavior that meets two criteria:students add task content to the dialogue (i.e., they construct) and their construction builds off their partners contributions.Unfortunately, co-construction is uncommon. In our studies of students collaborating face-to-face in middle school mathclassrooms, less than 5% of their spoken dialogue was classified as co-construction. In order to increase the frequencyof co-construction and raise learning gains, prior work has inserted prompts into text-based dialogue, but our FACTsystem is alone in trying to use prompting to improve spoken dialogues in classrooms. Results on the accuracy of FACTscollaboration detectors will be presented along with results from a pilot test of its prompting in 5 middle school classrooms.

Effective collaboration between student peers is not spontaneous. A system that can measure collaboration in real-time may be useful, as it could alert an instructor to pairs that need help in collaborating effectively. We tested whether superficial measures of speech and user interface actions would suffice for measuring collaboration. Pairs of students solved complex math problems while data were collected in the form of verbal interaction and user action logs from the students' tablets. We distinguished four classifications of interactivity: collaboration, cooperation, high asymmetric contribution and low asymmetric contribution. Human coders used richer data (several video streams) to choose one of these codes for each episode. Thousands of features were extracted computationally from the log and audio data. Machine learning was used to induce a detector that also assigned a code to each episode as a function of these features. Detectors for combinations of codes were induced as well. The best detector's overall accuracy was 96 percent (kappa = 0.92) compared to human coding. This high level of agreement suggests that superficial features of speech and log data do suffice for measuring collaboration. However, these results should be viewed as preliminary because the particular task may have made it relatively easy to distinguish collaboration from cooperation.

Background: The Office of Naval Research (ONR) organized a STEM Challenge initiative to explore how intelligent tutoring systems (ITSs) can be developed in a reasonable amount of time to help students learn STEM topics. This competitive initiative sponsored four teams that separately developed systems that covered topics in mathematics, electronics, and dynamical systems. After the teams shared their progress at the conclusion of an 18-month period, the ONR decided to fund a joint applied project in the Navy that integrated those systems on the subject matter of electronic circuits. The University of Memphis took the lead in integrating these systems in an intelligent tutoring system called "ElectronixTutor." This article describes the architecture of ElectronixTutor, the learning resources that feed into it, and the empirical findings that support the effectiveness of its constituent ITS learning resources. Results: A fully integrated ElectronixTutor was developed that included several intelligent learning resources (AutoTutor, Dragoon, LearnForm, ASSISTments, BEETLE-II) as well as texts and videos. The architecture includes a student model that has (a) a common set of knowledge components on electronic circuits to which individual learning resources contribute and (b) a record of student performance on the knowledge components as well as a set of cognitive and non-cognitive attributes. There is a recommender system that uses the student model to guide the student on a small set of sensible next steps in their training. The individual components of ElectronixTutor have shown learning gains in previous decades of research. Conclusions: The ElectronixTutor system successfully combines multiple empirically based components into one system to teach a STEM topic (electronics) to students. A prototype of this intelligent tutoring system has been developed and is currently being tested. ElectronixTutor is unique in its assembling a group of well-tested intelligent tutoring systems into a single integrated learning environment.

This paper describes Dragoon, a simple intelligent tutoring system which teaches the construction of models of dynamic systems. Modelling is one of seven practices dictated in two new sets of educational standards in the U.S.A., and Dragoon is one of the first systems for teaching model construction for dynamic systems. Dragoon can be classified as a step-based tutoring system that uses example-tracing, an explicit pedagogical policy and an open learner model. Dragoon can also be used for computer-supported collaborative learning, and provides tools for classroom orchestration. This paper describes the features, user interfaces, and architecture of Dragoon; compares and contrasts Dragoon with other intelligent tutoring systems; and presents a brief overview of formative and summative evaluations of Dragoon in both high school and college classes. Of four summative evaluations, three found that students who used Dragoon learned more about the target system than students who did equivalent work without Dragoon.

The paper describes a biology tutoring system with adaptive question selection. Questions were selected for presentation to the student based on their utilities, which were estimated from the chance that the student's competence would increase if the questions were asked. Competence was represented by the probability of mastery of a set of biology knowledge components. Tasks were represented and selected based on which knowledge components they addressed. Unlike earlier work, where the knowledge components and their relationships to the questions were defined by domain experts, this project demonstrated that the knowledge components, questions and their relationships could all be generated from a semantic network. An experiment found that students using our adaptive question selection had reliably larger learning gains than students who received questions in a mal-adaptive order.

This project aimed to improve students' learning and task performance using a non-cognitive learning companion in the context of both a tutor and a meta-tutor. The tutor taught students how to construct models of dynamic systems and the meta-tutor taught students a learning strategy. The non-cognitive learning companion was designed to increase students' effort and persistence in using the learning strategy. It decided when to intervene and what to say using both log data and affective state monitoring via a facial expression camera and a posture sensor. Experiments with high school students showed that the non-cognitive learning companion increased students' learning and performance. However, it had no effect on performance during a transfer phase in which the learning companion, meta-tutor, and tutor were all absent. The transfer phase null effect must be interpreted with caution due to low power, a possible floor effect, and other issues.

Constructing models of dynamic systems is an important skill in both mathematics and science instruction. However, it has proved difficult to teach. Dragoon is an intelligent tutoring system intended to quickly and effectively teach this important skill. This paper describes Dragoon and an evaluation of it. The evaluation randomly assigned students in a university class to either Dragoon or baseline instruction that used Dragoon as an editor only. Among students who did use their systems, the tutored students scored reliably higher (p < 0.021, d = 1.06) on the post-test than the students who used only the conventional editor-based instruction.

A common hypothesis is that students will more deeply understand dynamic systems and other complex phenomena if they construct computational models of them. Attempts to demonstrate the advantages of model construction have been stymied by the long time required for students to acquire skill in model construction. In order to make model construction a feasible vehicle for science instruction, the Dragoon system combined three simplifications: (1) a simple notation for models of dynamic systems, (2) a step-based tutoring system, and (3) problems that described the model to be constructed as well as the system represented by the model. In order to test whether these simplifications reduced the time for learning how to construct models while preserving the benefits of model construction over baseline instruction, three classroom studies were conducted. All studies were experiments, in that they compared classes using Dragoon to classes learning the same material without Dragoon. However, as classroom studies, they could not tightly control all sources of variation. The first study produced null results, but it compared learning across just one class period. The second study in 4 high school science classes showed that instruction based on Dragoon cost only one extra class period (about 50 min) out of 4 class periods and was more effective than the same content taught without Dragoon. A third study in 3 more high school science classes, where 2 Dragoon classes and 1 non-Dragoon class met for the same number of class periods, showed that Dragoon was more effective than the same content taught without Dragoon. The effect sizes were moderately large on both an open response test (d = 1.00) and a concept mapping task (d = 0.49). Thus, it appears that our efforts have simplified model construction to the point that it can be used in science instruction with no additional class time needed, and yet it still seems to be more effective than the same instruction done without model construction.

This commentary suggests a generalization of the conception of the behavior of tutoring systems, which the target article characterized as having an outer loop that was executed once per task and an inner loop that was executed once per step of the task. A more general conception sees these two loops as instances of regulative loops, which repeatedly compare students' performances to a gold standard and give advice on how to bring that performance closer to the standard.

Although the Andes project produced many results over its 18 years of activity, this commentary focuses on its contributions to understanding how a goal-free user interface impacts the overall design and performance of a step-based tutoring system. Whereas a goal-aligned user interface displays relevant goals as blank boxes or empty locations that the student needs to fill with specific content, a goal-free user interface is essentially a blank canvas, with no visual indications of the goals that should be attempted next. This commentary also briefly mentions work that occurred after the "final" report on Andes appeared in this journal in 2005. The newer work focused on getting students to ask for hints when they need them.

Science instructors need questions for use in exams, homework assignments, class discussions, reviews, and other instructional activities. Textbooks never have enough questions, so instructors must find them from other sources or generate their own questions. In order to supply biology instructors with questions for college students in introductory biology classes, two algorithms were developed. One generates questions from a formal representation of photosynthesis knowledge. The other collects biology questions from the web. The questions generated by these two methods were compared to questions from biology textbooks. Human students rated questions for their relevance, fluency, ambiguity, pedagogy, and depth. Questions were also rated by the authors according to the topic of the questions. Although the exact pattern of results depends on analytic assumptions, it appears that there is little difference in the pedagogical benefits of each class, but the questions generated from the knowledge base may be shallower than questions written by professionals. This suggests that all three types of questions may work equally well for helping students to learn.