Search

Homework is considered as a substantial process of learning especially for engineering education. However, due to the fast development of network technology, students now can easily find solution manuals on the internet. While some students use solution manuals to study, there are quite a few students who just copy homework solutions and lose opportunities to learn. This paper proposes a self-assessment based homework model. In addition to finishing homework students need to correct their homework by using solutions and assess their performance using autopsies designed by the instructor. To test the efficiency of this model, an experimental study was carried out in "Control of Mechanical System" class with an enrollment of 39 students at California State University, Los Angeles. The results suggest that the proposed homework assignment model is effective.

This study examines the extent to which instructions to self-explain vs. "other"-explain a text lead readers to produce different forms of explanations. Natural language processing was used to examine the content and characteristics of the explanations produced as a function of instruction condition. Undergraduate students (n = 146) typed either self-explanations or other-explanations while reading a science text. The linguistic properties of these explanations were calculated using three automated text analysis tools. Machine learning classifiers in combination with the features were used to predict instruction condition (i.e., self- or other explanation). The best machine learning model performed at rates above chance (kappa = 0.247; accuracy = 63%). Follow-up analyses indicated that students in the self-explanation condition generated explanations that were more cohesive and that contained words that were more related to social order (e.g., ethics). Overall, the results suggest that natural language processing techniques can be used to detect subtle differences in students' processing of complex texts. [This paper was published in: "Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK19)" (Article 4 p116-120). New York, NY: ACM.]

Lecture material of a sophomore large-enrollment (N=271) system programming 15-week class was delivered solely online using a new video-based web platform. The platform provided accurate accessible transcriptions and captioning plus a custom text-searchable interface to rapidly find relevant video moments from the entire course. The system logged student searching and viewing behaviors as fine-grained web browser interaction events including full-screen-switching, loss-of-focus, incremental searching events, and continued-video-watching events with the latter at 15-second granularity. Student learning behaviors and findings from three research questions are presented using individual-level performance and interaction data. Firstly, we report on learning outcomes from alternative learning paths that arise from the course's application of Universal Design for Learning principles. Secondly, final exam performance was equal or better to prior semesters that utilized traditional in-person live lectures. Thirdly, learning outcomes of low and high performing students were analyzed independently by grouping students into four quartiles based on their non-final-exam course performance of programming assignments and quizzes. We introduce and justify an empirically-defined qualification threshold for sufficient video minutes viewed for each group. In all quartiles, students who watched an above-threshold of video minutes improved their in-group final exam performance (ranging from +6% to +14%) with the largest gain for the lowest-performing quartile. The improvement was similar in magnitude for all groups when expressed as a fraction of unrewarded final exam points. Overall, the study presents and evaluates how learner use of online video using ClassTranscribe predicts course performance and positive learning outcomes. [This paper was published in "SIGCSE '20" (Portland, Oregon, March 11-14, 2020). ACM. (ISBN 978-1-4503-6793-6)]

This demonstration introduces and presents an innovative online cognitive diagnostic assessment, developed to identify the types of cognitive processes that readers use during comprehension; specifically, processes that distinguish between subtypes of struggling comprehenders. Cognitive diagnostic assessments are designed to provide valuable information by measuring specific processes emphasized during learning, and can provide instructionally relevant results aligned with curriculum that other large-scale, standardized assessments cannot provide. This hands-on session includes in-formation behind how the technology of MOCCA was developed, as well as how a reader would experience taking this assessment, how a teacher/educator would find the results of a user's assessment, and which instructional techniques to then use. Interpretation of assessment results and instructional recommendations are obtainable online. Future directions for the continued development of online digital learning regarding how to generate appropriate cognitive processes (e.g., inferences) during reading are ongoing and discussed. [This paper was published in: "EC-TEL Proceedings," LNCS 11722, 2019, pp. 769-772.]

This study investigated the effect of incorporating spelling and grammar checking tools within an automated writing tutoring system, Writing Pal. High school students (n = 119) wrote and revised six persuasive essays. After initial drafts, all students received formative feedback about writing strategies. Half of the participants were also given access to spelling and grammar checking tools during the writing and revision periods. Linear mixed effects models revealed that essay quality for students in both conditions improved from initial draft to revision in terms of all aspects except essay unity. The availability of spelling and grammar checking yielded added improvements from initial draft to revision for several aspects of essay quality (i.e., conclusion, organization, voice, grammar/mechanics, and word choice), but other aspects were unaffected (i.e., introduction, body, unity, and sentence structure). The availability of spelling and grammar checking tools had no effect on holistic essay scores. These results indicate that automated spelling and grammar feedback contribute to modest, incremental improvements in writing quality that may complement automated strategy feedback. [This paper was published in: S. Isotani et al. (Eds.), "AIED 2019" (pp. 270-282). Switzerland: Springer.]

Summarization is an effective strategy to promote and enhance learning and deep comprehension of texts. However, summarization is seldom implemented by teachers in classrooms because the manual evaluation of students' summaries requires time and effort. This problem has led to the development of automated models of summarization quality. However, these models often rely on features derived from expert ratings of student summarizations of specific source texts and are therefore not generalizable to summarizations of new texts. Further, many of the models rely of proprietary tools that are not freely or publicly available, rendering replications difficult. In this study, we introduce an automated summarization evaluation (ASE) model that depends strictly on features of the source text or the summary, allowing for a purely textbased model of quality. This model effectively classifies summaries as either low or high quality with an accuracy above 80%. Importantly, the model was developed on a large number of source texts allowing for generalizability across texts. Further, the features used in this study are freely and publicly available affording replication. [This paper was published in: S. Isotani et al. (Eds.), "AIED 2019" (pp. 84-95). Switzerland: Springer.]

This study evaluated the efficacy of the Word Learning Strategies (WLS) supplementary program to improve elementary students' vocabulary skills and reading comprehension. The study used a multi-site cluster randomized, experimental design, which randomly assigned 92 4th grade classrooms (n=2558 students) from two cohorts to a treatment or control group. Results indicated that the program was positively associated with gains in students' vocabulary learning and knowledge as measured by the Word Learning Strategies Test and the VASE Assessment, and in students' reading comprehension as measured by the Gates-MacGrinitie Reading Test, after accounting for differences in baseline measures. The use of the WLS program also led to increases in teachers' awareness of strategies to support their students' vocabulary and reading comprehension. [This paper was published in the Proceedings of the 2019 Annual Meeting of the American Educational Research Association (AERA), Toronto, Canada.]

We study the usage of a self-guided online tutoring platform called Algebra Nation, which is widely by middle school and high school students who take the End-of-Course Algebra I exam at the end of the school year. This article aims to study how the platform contributes to increasing students' exam scores by examining users' logs over a three year period. The platform under consideration was used by more than 36,000 students in the first year, to nearly 67,000 by the third year, thus enabling us to examine how usage patterns evolved and influenced students' performance at scale. We first identify which Algebra Nation usage factors in conjunction with math overall preparation and socioeconomic factors contribute to the students' exam performance. Subsequently, we investigate the effect of increased teacher familiarity level with the Algebra Nation on students' scores across different grades through mediation analysis. The results show that the indirect effect of teacher's familiarity with the platform through increasing student's usage dosage is more significant in higher grades. [This paper was published in: "The 9th International Learning Analytics & Knowledge Conference (LAK19)" (Tempe, AZ, March 4-8, 2019). New York, NY: Association for Computing Machinery (ACM). (978-1-4503-6256-6).]

Mission HydroSci (MHS) teaches water systems and scientific argumentation towards meeting Next Generation Science Standards. MHS is a game-based 3D virtual environment for enacting transformational role-playing, wherein students must learn new knowledge and competencies in order to successfully complete the game missions. MHS was developed for middle school science as a replacement unit of about 6 to 8 hours and uses analytics and a teacher dashboard to help support teachers support their students. [This paper was published in: "CSCL 2019 Proceedings," p.1017-1020. ISLS.]

Prior studies on scaffolding for investigative inquiry practices (i.e. forming a question/hypothesis, collecting data, and analyzing and interpreting data [21]) revealed that students who received scaffolding were better able to both learn practices and transfer these competencies to new topics than were students who did not receive scaffolding. Prior studies have also shown that after removing scaffolding, students continued to demonstrate improved inquiry performance on a variety of practices across new driving questions over time. However, studies have not examined the relationship between the "amount of scaffolding received" and transfer of inquiry performance; this is the focus of the present study. 107 middle school students completed four virtual lab activities (i.e. driving questions) in Inq-ITS. Students received scaffolding when needed from an animated pedagogical computer agent for the first three driving questions for the Animal Cell virtual lab. Then they completed the fourth driving question without access to scaffolding in a different topic, Plant Cell. Results showed that students' performances increased even with fewer scaffolds for the inquiry practices of hypothesizing, collecting data, interpreting data, and warranting claims; furthermore, these results were robust as evidenced by the finding that students required less scaffolding as they completed subsequent inquiry activities. These data provide evidence of near and far transfer as a result of adaptive scaffolding of science inquiry practices. [This paper was published in: "Proceedings of the Sixth Annual ACM Conference on Learning at Scale."]

Understanding equivalence is fundamental to STEM disciplines, but many students struggle with the concept. We present a novel method for students to explore ideas of equivalence where students dynamically transform expressions from initial states to goal states in the web-based app Graspable Math. The structure of the goal state activities implies that any initial and goal state pair represent the same quantity (or varying quantity). We propose that for students, the physical experience of moving algebraic objects and observing how the initial state transforms into the goal state helps generalize notation mechanics. In fall of 2019, we will test this supposition in a randomized control trial of 1500 students in which student performance in pre- and post-tests will be compared to an online problem set control. [This paper was published in: "Proceedings of the forty-first annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education," edited by S. Otten et al., University of Missouri, 2019, pp. 1738-43.]

Educational technologies with multimedia content can support effective learning, but these outcomes are moderated by learners' level of cognitive engagement or self-regulation. As a way to encourage deeper cognitive engagement without redesigning or redeveloping software (e.g., building more prompts, scaffolds, or automated support), this study investigates changing the "role" of the student user. Specifically, this research considers how instituting a "designer" or "teacher" role may inspire better engagement and learning than the default "learner" role. We present the theoretical background, design, and results of an exploratory study of this hypothesis with college students learning about cohesion in writing. [This paper was published in: "Proceedings of the Human Factors and Ergonomics Society 2018 Annual Meeting," 2018, pp. 2074-2078.]

This paper discusses how a dialogue-based tutoring system makes decisions to proactively scaffold students during conceptual discussions about physics. The tutor uses a student model to predict the likelihood that the student will answer the next question in a dialogue script correctly. Based on these predictions, the tutor will, step by step, choose the granularity at which the next step in the dialogue is discussed. The tutor attempts to pursue the discussion at the highest possible level, with the goal of helping the student achieve mastery, but with the constraint that the questions it asks are within the student's ability to answer when appropriately supported; that is, the tutor aims to stay within its estimate of the student's zone of proximal development for the targeted concepts. The scaffolding provided by the tutor is further adapted by adjusting the way the questions are expressed.

Examining the interaction between content knowledge, inquiry proficiency, and writing proficiency is central to understanding the relative contribution of each proficiency on students' written communication about their science inquiry. Previous studies, however, have only analyzed one of these primary types of knowledge/proficiencies (i.e. content knowledge, inquiry proficiency, and writing proficiency) at a time. This study investigated the extent to which these proficiencies predicted students' written claims, evidence for their claims, and reasoning linking their claims to the evidence. Results showed that all three types of proficiencies significantly predicted students' claims, but only writing proficiency significantly predicted performance on evidence and reasoning statements. These findings indicate the challenges students face when constructing claim, evidence, and reasoning statements, and can inform scaffolding to support these challenges. [This paper was published in: "L@S '18: Proceedings of the Fifth Annual ACM Conference on Learning at Scale." ACM. (978-1-4503-5886-6).]

Automated essay scoring tools are often criticized on the basis of construct validity. Specifically, it has been argued that computational scoring algorithms may be unaligned to higher-level indicators of quality writing, such as writers' demonstrated knowledge and understanding of the essay topics. In this paper, we consider how and whether the scoring algorithms within an intelligent writing tutor correlate with measures of writing proficiency and students' general knowledge, reading comprehension, and vocabulary skill. Results indicate that the computational algorithms, although less attuned to knowledge and comprehension factors than human raters, were marginally related to such variables. Implications for improving automated scoring and intelligent tutoring of writing are briefly discussed. [This paper was published in: "Proceedings of the Twenty-Seventh International Florida Artificial Intelligence Research Society Conference" (p.393-398). Association for the Advancement of Artificial Intelligence, 2014.]

Much research has been done on the development of an intelligent tutoring system (ITS), and small empirical studies have demonstrated the effectiveness of ITS at promoting student learning. However, large-scale implementation of ITS in school settings has not been researched thoroughly. In this paper, we describe an ongoing randomized controlled trial (RCT) to evaluate the efficacy of a web-based tutoring system--the ASSISTments--as support for homework. The program is used in 46 middle schools in the state of Maine, to provide immediate feedback to students, and to provide reports to teachers to support homework review and instruction adaptation. We describe the challenges for the RCT, approaches used to understand implementation of the system, and findings on how the system is being used. [This paper was published in: Proceedings of ITS 2014, Honolulu, HI, USA, June 5-9, 2014. p561-566.]

Authoring tools for Intelligent Tutoring System (ITS) have been shown to decrease the amount of time that it takes to develop an ITS. However, most of these tools currently do not extend to collaborative ITSs. In this paper, we illustrate an extension to the Cognitive Tutor Authoring Tools (CTAT) to allow for development of collaborative ITSs that can support a range of collaboration scripts. Authoring tools for collaborative ITSs must be flexible enough to allow for different learning goals and different collaboration scripts. We discuss how two collaboration scripts that we are using in our research on fractions learning are implemented in CTAT. The examples illustrate how CTAT flexibly supports collaborative tutors by running synchronized tutor engines for each student, and how it supports the development of collaborative tutors through the use of multiple behavior graphs that use no programming to develop. [This paper was published in: "Proceedings of the Workshops at the 16th International Conference on Artificial Intelligence in Education AIED 2013 Memphis, USA, July 9-13, 2013," Vol. 1009, edited by Erin Walker and Chee-Kit Looi, CEUR-WS.org, 2013, pp. 1-10.]

A critical issue in using fine-grained gameplay data to measure learning processes is the development of indicators and the algorithms used to derive such indicators. Successful development--that is, developing traceable, interpretable, and sensitive-to-learning indicators--requires understanding the underlying theory, how the theory is instantiated under different game conditions, and programming that considers these exigencies using fine-grained gameplay data. This study examines preliminary validity evidence on various game-based indicators (GBIs) developed from theoretical frameworks and not from unsupervised feature extraction methods. Examples of indicators are presented that represent three types of indicators: Common indicators, distance-based indicators, and games-specific indicators. For each example, the theoretical background is presented briefly, followed by a description of how the indicator design flows from the theory. Correlational analyses show how the GBIs relate to external criterion measures. Limitations and next steps are discussed.

As implementation of the "Next Generation Science Standards" moves forward, there is a need for new assessments that can measure students' integrated three-dimensional science learning. The National Research Council has suggested that these assessments be multicomponent tasks that utilize a combination of item formats including constructed-response and multiple-choice. However, little guidance has been provided for determining the relative value or cost effectiveness of those two formats. In this study, students were randomly assigned assessment tasks that contained either a constructed-response or a multiple-choice version of an otherwise equivalent item. Rasch analysis was used to compare the difficulty of these items on the same construct scale. We found that the set of items formed a broad unidimensional scale, but the constructed-response versions were more difficult than their multiple-choice counterparts. This difficulty was found to be partially due to the increased writing demand and the reasoning element in the constructed-response rubric. Students were more likely to recognize a clearly reasoned argument in a multiple-choice item than they were to create that reasoning themselves and communicate it in writing. Our findings can help instrument developers select a set of items that balances the time and effort students must provide during testing and the time and effort scorers need to spend to evaluate and score students' responses. In cases where constructing a response is an essential part of the targeted understanding, as when the target learning goal is to be able to construct an argument or generate a model, CR items are needed, but in other cases, MC items may be more efficient.

Large language models have recently been able to perform well in a wide variety of circumstances. In this work, we explore the possibility of large language models, specifically GPT-3, to write explanations for middle-school mathematics problems, with the goal of eventually using this process to rapidly generate explanations for the mathematics problems of new curricula as they emerge, shortening the time to integrate new curricula into online learning platforms. To generate explanations, two approaches were taken. The first approach attempted to summarize the salient advice in tutoring chat logs between students and live tutors. The second approach attempted to generate explanations using few-shot learning from explanations written by teachers for similar mathematics problems. After explanations were generated, a survey was used to compare their quality to that of explanations written by teachers. We test our methodology using the GPT-3 language model. Ultimately, the synthetic explanations were unable to outperform teacher written explanations. In the future more powerful large language models may be employed, and GPT-3 may still be effective as a tool to augment teachers' process for writing explanations, rather than as a tool to replace them. The explanations, survey results, analysis code, and a dataset of tutoring chat logs are all available at https://osf.io/wh5n9/.

Oral reading fluency (ORF) assessments are commonly used to screen at-risk readers and to evaluate the effectiveness of interventions as curriculum-based measurements. As with other assessments, equating ORF scores becomes necessary when we want to compare ORF scores from different test forms. Recently, Kara et al. (2023) proposed a model-based equating method for ORF scores. However, they did not provide closed-form asymptotic standard errors (SEs) of the equated ORF scores while it is advocated to report SEs of equating in practice. Therefore, this study aims to address this remaining question. Specifically, the delta method was adopted to derive the asymptotic SEs of equated ORF scores. The ORF scoring was conducted using an approach that takes into account the calibration error. Its performance was compared to the standard practice that ignores calibration errors. It is expected that the two scoring approaches would further have impact on the recovery of equated ORF score SEs. A simulation study was conducted to evaluate the recovery of derived equating SEs in various conditions. Results suggested satisfactory recovery of the derived asymptotic SEs of equated ORF scores. In addition, taking into account the calibration error in ORF scoring can produce more accurate and precise equated ORF score SEs under larger sample sizes and longer test lengths.

The central aim of the EMIC Working Group is to connect, engage, and inspire colleagues in this growing community of discourse around theoretical, technological, and methodological developments for advancing the study of embodied cognition for mathematics education. This year, our fifth at PME-NA, we also will convene on Day 3 with the WG on Mathematical Play. Our community of scholars will use these sessions to continue to broaden the range of activities, practices, and emerging technologies that contribute to mathematics teaching and learning as well as to research on these phenomena. [This paper was published in: "Mathematics Education Across Cultures: Proceedings of the 42nd Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education," edited by A. I. Sacristán, et al., Cinvestav / AMIUTEM / PME-NA, 2020, pp. 166-168.]

Novel forms of technology, like shared Augmented Reality (AR) holograms, can spur the discovery of new hypotheses about cognition and how it is embodied and distributed. These holograms have affordances for exploration, collaboration, and learning that have never been seen before. In the present study, we examine the multimodal ways that high school students interact with shared AR holograms while exploring geometric conjectures. Through multimodal analysis, we find both possibilities and important design considerations. [This paper was published in: "ICLS Proceedings," International Society of the Learning Sciences, 2022, pp. 1992-1993.]

Diagrams are powerful opportunities for grappling with and learning abstract relationships, for example learning the relations between elements in an ecosystem rather than simply memorizing the objects within the system. Further, what is crucial from any diagrammatic learning opportunity is the ability to use this relational knowledge in a new context or with new materials, beyond simply understanding the initial presentation. This is cognitively effortful, however, and individual differences in how reasoners benefit from such relational learning opportunities are not well understood. We describe a computational simulation that examines how cognitive control of attention enables relational learning from visual stimuli such as diagrams. Specifically, we propose that cognitive control is critical for both abstracting relational representations from that visual stimuli, and to the ability to use these representations in subsequent problem solving. [This paper was published in: Chapman P., Stapleton G., Moktefi A., Perez-Kriz S., Bellucci F. (eds) "Diagrammatic Representation and Inference. Diagrams 2018: LNAI 10871" (pp. 821-824). Edinburgh, UK: Springer.]

Learning from expository science texts is challenging. These studies explore whether difficulties can be attributed to poor memory or poor reasoning. To eliminate the need for memory during testing, some students took the tests with the texts available. To test for the effects of reasoning on performance, some students were prompted to engage in explanation activities during or after reading. The effects of these manipulations were tested on text-based and inference questions. Allowing the reader access to the texts during testing improved performance for text-based questions. In contrast, engaging in explanation activities during reading improved performance on inference questions. These results suggest that achieving a better understanding from expository texts depends on engaging in constructive reasoning processes, and not simply improving memory for the texts. [The paper was published in: "Proceedings of the 40th Annual Conference of the Cognitive Science Society," p. 1745-1750. Madison, WI: Cognitive Science Society.]

This study evaluated the efficacy of the Word Learning Strategies (WLS) supplementary program designed to develop upper-elementary students' vocabulary skills in order to improve reading comprehension. The study used a true, group-randomized, experimental design, which randomly assigned 46 4th grade classrooms (n=1324 students) from 12 districts to a treatment or control group. The results from the first cohort indicate that the program was positively associated with gains in students' vocabulary learning as measured by Word Learning Strategies Test and in students' reading comprehension as measured by Gates-MacGinitie Reading Test, after accounting for differences in baseline test scores. The use of the WLS program also led to increases in teachers' awareness of strategies to support their students' vocabulary and reading comprehension. [This paper was published in the Proceedings of the 2018 Annual Meeting of the American Educational Research Association (AERA), New York, NY.]

Gestures have been shown to play a key role in mathematical reasoning and be an indicator that mathematical reasoning is "embodied" -- inexorably linked to action, perception, and the physical body. Theories of extended cognition accentuate looking beyond the body and mind of an individual, thus here we examine how gestural embodied actions become distributed over multiple learners confronting mathematical tasks. We identify several ways in which gesture can be used collaboratively and explore patterns in how collaborative gestures seem to arise in a learning environment involving a motion capture game for geometry. Learners use collaborative gestures to extend mathematical ideas over multiple bodies as they explore, refine, and extend each other's reasoning. [This paper was published in: "ICLS 2018 Proceedings," p552-559. ISLS.]

Most community college students in the U.S. must complete at least one developmental class, such as elementary algebra, before they can enroll in a college-level mathematics course. Increasingly common in such courses is the use of a web-based activity and testing system (WATS). This report presents initial results of a mixed-methods study of elementary algebra learning among 510 students in the classes of 29 instructors across 18 community colleges. Instructors were randomly assigned to use a particular WATS (treatment condition) or their usual approach (control condition). The focal WATS had adaptive problem sets, hints, and videos. Treatment group instructors had access to online support for implementation. For the study, students completed common pre- and post-tests and instructors regularly provided information about their teaching practices. The early results reported here indicate that greater instructor fidelity to developer intentions regarding frequency of assignments are positively associated with greater student learning. [This paper was published in: A. Weinberg, C. Rasmussen, J. Rabin, M. Wawro, and S. Brown (Eds.), "Proceedings of the 21st Annual Conference on Research in Undergraduate Mathematics Education, San Diego, California," 2018.]

In the context of a large-scale randomized controlled trial, our team investigated "front line" teaching issues as schools implemented a fully digital, blended learning curriculum in mathematics. This paper focuses on observations of instruction within schools that were assigned to use the new digital resources. Compared to a business-as-usual control group, classroom activity and teaching practices changed in the treatment group. Observers, who were blind to student achievement outcomes, found two overall patterns in treatment classrooms across five categories of observations. Later quantitative analysis indeed found the "high" and "low" patterns could account for some of the variance in achievement outcomes within the treatment condition. We explore observed patterns in terms of existing learning science theory and suggest areas where further development of the learning sciences may be needed and how learning sciences can contribute to improvement of digital, blended learning environments.

Language is often depicted as the sine qua non of mathematical thinking, a view buttressed by findings of language-of-training effects among bilinguals. These findings, however, have been limited to studies of arithmetic. Nothing is known about the potential influence of language on the ability to learn rules about the relations among variables (e.g., algebra). To test whether arithmetic and algebraic thinking differ, Chinese-English bilinguals were trained to solve arithmetic and algebra problems in either Chinese or English and then tested on new and old problems in both languages. For arithmetic problems, solution times were always longer for English than Chinese; in both languages, solution times dropped during training; after training, solution times continued to drop for old problems, but returned to pre-training levels for new problems. In contrast, for algebra problems, solution times did not differ across language; solution times dropped during training; after training, gains in speed were preserved for both old and new problems. These findings suggest that the contribution of language to mathematical thinking may be limited to the areas of mathematics that are learned by rote and not by rule. [This paper was published in: "Proceedings of the 40th Annual Conference of the Cognitive Science Society," edited by T. T. Rogers et al., Cognitive Science Society, 2018, pp. 918-923.]

A common goal of Intelligent Tutoring Systems (ITS) is to provide learning environments that adapt to the varying abilities and characteristics of users. To do this, researchers must identify the learning patterns exhibited by those interacting with the system. In the present work, we use clustering analysis to capture learning patterns in over 250 adults who used the ITS, "CSAL" (Center for the Study of Adult Literacy) "AutoTutor," to gain reading comprehension skills. AutoTutor has conversational agents that teach literacy adults with low literacy skills comprehension strategies in 35 lessons. These comprehension strategies align with one or more of the following levels specified in the Graesser-McNamara theoretical framework of comprehension: "word," "textbase," "situation model" and "rhetorical structure." We used the adult learners' average response times per question and performance across lessons to cluster the students' learning behavior. Performance was measured as the proportion of 3-alternative-response questions answered correctly. Lessons were coded on one of the four theoretical levels of comprehension. Results of the cluster analyses converged on four types of learners: proficient readers, struggling readers, conscientious readers and disengaged readers. Proficient readers were fast and accurate; struggling readers worked slowly but were not accurate; conscientious readers worked slowly and performed comparatively well; disengaged readers were fast but did not perform well. Interestingly, the behaviors of learners in different clusters varied across the four theoretical levels. Identifying types of readers can enhance the adaptivity of AutoTutor by allowing for more personalized feedback and interventions designed for particular learning behaviors. [This paper was published in: K.E. Boyer & M. Yudelson (Eds.), "Proceedings of the 11th International Conference on Educational Data Mining" (pp.348-354). Buffalo, NY: Educational Data Mining Society.]

This study assesses the extent to which machine learning techniques can be used to predict question quality. An algorithm based on textual complexity indices was previously developed to assess question quality to provide feedback on questions generated by students within iSTART (an intelligent tutoring system that teaches reading strategies). In this study, 4,575 questions were coded by human raters based on their corresponding depth, classifying questions into four categories: 1-very shallow to 4-very deep. Here we propose a novel approach to assessing question quality within this dataset based on Recurrent Neural Networks (RNNs) and word embeddings. The experiments evaluated multiple RNN architectures using GRU, BiGRU and LSTM cell types of different sizes, and different word embeddings (i.e., FastText and Glove). The most precise model achieved a classification accuracy of 81.22%, which surpasses the previous prediction results using lexical sophistication complexity indices (accuracy = 41.6%). These results are promising and have implications for the future development of automated assessment tools within computer-based learning environments. [This is a paper in: Penstein Rosé, C., et al. "Artificial Intelligence in Education." AIED 2018. Lecture Notes in Computer Science(), vol 10947. Springer, Cham.]

Cloze items are a foundational approach to assessing readability. However, they require human data collection, thus making them impractical in automated metrics. The present study revisits the idea of assessing readability with cloze items and compares human cloze scores and readability judgments with predictions made by T5, a popular deep learning architecture, on three corpora. Across all corpora, T5 predictions significantly correlated with human cloze scores and readability judgments, and in predictive models, they could be used interchangeably with average word length, a common readability predictor. For two corpora, combining T5 and Flesch reading ease predictors improved model fit for human cloze scores and readability judgments.