Search

Archived transcripts from tens of millions of online human tutoring sessions potentially contain important knowledge about how online tutors help, or fail to help, students learn. However, without ways of automatically analyzing these large corpora, any knowledge in this data will remain buried. One way to approach this issue is to train an estimator for the learning effectiveness of an online tutoring interaction. While significant work has been done on automated assessment of student responses and artifacts (e.g., essays), automated assessment has not traditionally automated assessments of human-to-human tutoring sessions. In this work, we trained a model for estimating tutoring session quality based on a corpus of 1438 online tutoring sessions rated by expert tutors. Each session was rated for evidence of learning (outcomes) and educational soundness (process). Session features for this model included dialog act classifications, mode classifications (e.g., Scaffolding), statistically distinctive subsequences of such classifications, dialog initiative (e.g., statements by tutor vs. student), and session length. The model correlated more highly with evidence of learning than educational soundness ratings, in part due to the greater difficulty of classifying tutoring modes. This model was then applied to a corpus of 242k online tutoring sessions, to examine the relationships between automated assessments and other available metadata (e.g., the tutor's self-assessment). On this large corpus, the automated assessments followed similar patterns as the expert rater's assessments, but with lower overall correlation strength. Based on the analyses presented, the assessment model for online tutoring sessions emulates the ratings of expert human tutors for session quality ratings with a reasonable degree of accuracy. [For complete proceedings, see ED560503.]

The present paper describes the recruitment and site selection process that has been underway since January 2011, with particular emphasis on the use of Mahalanobis distance score to determine matched pairs of sites prior to randomization to treatment and control groups. Through a systematic winnowing process, the authors found that they could reduce the number of potential matching variables from more than 100 to just 11 matching variables. By computing Mahalanobis Distance scores for each eligible school, and discussing the results as a team, they were able to identify a set of matched pairs that, when randomly assigned to their two treatment conditions, produced two sets of schools that were well matched on the relevant variables, i.e., represented a level playing field for the RCT in North Carolina. They will use the same process to match and randomly assign schools in their other two regions. They hope that other researchers engaged in similar work will find this account useful. (Contains 5 tables.)

Scientific sense-making is a process in which theory and evidence are brought into coordination. This paper reports on research undertaken as part of a larger National Science Foundation funded study on the conditions that support scientific sense-making in schools. It describes how students with access to a range of epistemologies contribute in different ways to a conversation about a toy called a Magic Eight Ball. The investigators assert that conversations of this type may provide critical learning opportunities for students who are developing new ways of thinking about what it means to know something, and what it means to do science. This discussion is intended for use by teachers and science educators who wish to give scientific sense-making a more prominent role in the classroom than it presently enjoys. (ZWH)