Search

Past research on knowledge has differentiated between dimensions (e.g., amount, accuracy, specificity, coherence) of knowledge. This paper introduces a novel dimension of knowledge, the Motivational Utility of Knowledge (MUK), that is based on fundamental human needs (e.g., physical safety, affiliation, actualization, reproduction). Adults in the United States (N = 190) were recruited from an online survey platform and paid for participation. Participants read a set of four texts arguing different views of houselessness and were administered a comprehension test after each text. Participants were asked about their conceptions of houselessness before and after reading. Finally, they were given the MUK scale, a demographics questionnaire, including questions about their personal experience with houselessness, and were administered a general prior knowledge test and a vocabulary knowledge test. We examined MUK, the factor structure of the scale and the relationship between MUK and other measures of knowledge. The analyses showed that the subscales of MUK loaded onto a single factor—an overall value of houselessness knowledge. In addition, we found that MUK was correlated with conceptions of houselessness and comprehension of texts on houselessness, indicating that the scale was valid. Overall, the findings demonstrate that MUK is an important dimension of knowledge to consider in learning tasks.

The dramatic expansion of modern linguistic research and enhanced accuracy of linguistic analysis have become a reality due to the ability of artificial neural networks not only to learn and adapt, but also carry out automate linguistic analysis, select, modify and compare texts of various types and genres. The purpose of this article and the journal issue as a whole is to present modern areas of research in computational linguistics and linguistic complexology, as well as to define a solid rationale for the new interdisciplinary field, i.e. discourse complexology. The review of trends in computational linguistics focuses on the following aspects of research: applied problems and methods, computational linguistic resources, contribution of theoretical linguistics to computational linguistics, and the use of deep learning neural networks. The special issue also addresses the problem of objective and relative text complexity and its assessment. We focus on the two main approaches to linguistic complexity assessment: “parametric approach” and machine learning. The findings of the studies published in this special issue indicate a major contribution of computational linguistics to discourse complexology, including new algorithms developed to solve discourse complexology problems. The issue outlines the research areas of linguistic complexology and provides a framework to guide its further development including a design of a complexity matrix for texts of various types and genres, refining the list of complexity predictors, validating new complexity criteria, and expanding databases for natural language.

The study presents an overview of discursive complexology, an integral paradigm of linguistics, cognitive studies and computer linguistics aimed at defining discourse complexity. The article comprises three main parts, which successively outline views on the category of linguistic complexity, history of discursive complexology and modern methods of text complexity assessment. Distinguishing the concepts of linguistic complexity, text and discourse complexity, we recognize an absolute nature of text complexity assessment and relative nature of discourse complexity, determined by linguistic and cognitive abilities of a recipient. Founded in the 19th century, text complexity theory is still focused on defining and validating complexity predictors and criteria for text perception difficulty. We briefly characterize the five previous stages of discursive complexology: formative, classical, period of closed tests, constructive-cognitive and period of natural language processing. We also present the theoretical foundations of Coh-Metrix, an automatic analyzer, based on a five-level cognitive model of perception. Computing not only lexical and syntactic parameters, but also text level parameters, situational models and rhetorical structures, Coh-Metrix provides a high level of accuracy of discourse complexity assessment. We also show the benefits of natural language processing models and a wide range of application areas of text profilers and digital platforms such as LEXILE and ReaderBench. We view parametrization and development of complexity matrix of texts of various genres as the nearest prospect for the development of discursive complexology which may enable a higher accuracy of inter- and intra-linguistic contrastive studies, as well as automating selection and modification of texts for various pragmatic purposes.

Text comprehension is an essential skill in today’s information-rich world, and self-explanation practice helps students improve their understanding of complex texts. This study was centered on leveraging open-source Large Language Models (LLMs), specifically FLAN-T5, to automatically assess the comprehension strategies employed by readers while understanding Science, Technology, Engineering, and Mathematics (STEM) texts. The experiments relied on a corpus of three datasets (N = 11,833) with self-explanations annotated on 4 dimensions: 3 comprehension strategies (i.e., bridging, elaboration, and paraphrasing) and overall quality. Besides FLAN-T5, we also considered GPT3.5-turbo to establish a stronger baseline. Our experiments indicated that the performance improved with fine-tuning, having a larger LLM model, and providing examples via the prompt. Our best model considered a pretrained FLAN-T5 XXL model and obtained a weighted F1-score of 0.721, surpassing the 0.699 F1-score previously obtained using smaller models (i.e., RoBERTa).

Literacy assessment is essential for effective literacy instruction and training. However, traditional paper-based literacy assessments are typically decontextualized and may cause stress and anxiety for test takers. In contrast, serious games and game environments allow for the assessment of literacy in more authentic and engaging ways, which has some potential to increase the assessment’s validity and reliability. The primary objective of this study is to examine the feasibility of a novel approach for stealthily assessing literacy skills using games in an intelligent tutoring system (ITS) designed for reading comprehension strategy training. We investigated the degree to which learners’ game performance and enjoyment predicted their scores on standardized reading tests. Amazon Mechanical Turk participants (n = 211) played three games in iSTART and self-reported their level of game enjoyment after each game. Participants also completed the Gates–MacGinitie Reading Test (GMRT), which includes vocabulary knowledge and reading comprehension measures. The results indicated that participants’ performance in each game as well as the combined performance across all three games predicted their literacy skills. However, the relations between game enjoyment and literacy skills varied across games. These findings suggest the potential of leveraging serious games to assess students’ literacy skills and improve the adaptivity of game-based learning environments.

The goal of this study was to assess the relationships between computational approaches to analyzing constructed responses made during reading and individual differences in the foundational skills of reading in college readers. We also explored if these relationships were consistent across texts and samples collected at different institutions and texts. The study made use of archival data that involved college participants who produced typed constructed responses under thinking aloud instructions reading history and science texts. They also took assessments of vocabulary knowledge and proficiency in comprehension. The protocols were analyzed to assess two different ways to determine their cohesion. One approach involved assessing how readers established connections with themselves (i.e., to other constructed responses they produced). The other approach involved assessing connections between the constructed responses and the texts that were read. Additionally, the comparisons were made by assessing both lexical (i.e., word matching) and semantic (i.e., high dimensional semantic spaces) comparisons. The result showed that both approaches for analyzing cohesion and making the comparisons were correlated with vocabulary knowledge and comprehension proficiency. The implications of the results for theory and practice are discussed.

Academic discourse communities and learning circles are characterized by collaboration, sharing commonalities in terms of social interactions and language. The discourse of these communities is composed of jargon, common terminologies, and similarities in how they construe and communicate meaning. This study examines the extent to which discourse reveals “shared language” among its participants that can promote inclusion or affinity. Shared language is characterized in terms of linguistic features and lexical, syntactical, and semantic similarities. We leverage a multi-method approach, including (1) feature engineering using state-of-the-art natural language processing techniques to select the most appropriate features, (2) the bag-of-words classification model to predict linguistic similarity, (3) explainable AI using the local interpretable model-agnostic explanations to explain the model, and (4) a two-step cluster analysis to extract innate groupings between linguistic similarity and emotion. We found that linguistic similarity within and between the threaded discussions was significantly varied, revealing the dynamic and unconstrained nature of the discourse. Further, word choice moderately predicted linguistic similarity between posts within threaded discussions (accuracy = 0.73; F1-score = 0.67), revealing that discourse participants’ lexical choices effectively discriminate between posts in terms of similarity. Lastly, cluster analysis reveals profiles that are distinctly characterized in terms of linguistic similarity, trust, and affect. Our findings demonstrate the potential role of linguistic similarity in supporting social cohesion and affinity within online discourse communities.

Literacy is a critically important and contemporary issue for educators, scientists, and politicians. Efforts to overcome the challenges associated with illiteracy, and the subsequent development of literate societies, are closely related to those of poverty reduction and sustainable human development. In this paper, the authors examine literacy from the lens of text and discourse theorists who focus on the higher-order comprehension processes involved in literacy. Discourse processing models make the assumption that comprehension emerges from the construction of a mental model of the text, which relies on the reader generating inferences to connect ideas within the text and to what the reader already knows. The article provides a broad overview of the theoretical models that drive research on text comprehension and production, as well as how this research shapes literacy instruction and effective interventions. The authors focus on two interventions with proven success in improving deep comprehension and writing, iSTART and the Writing Pal. Increasing literacy across the world call for a greater focus on theory driven strategy interventions to be integrated within classrooms and community at large.

Learning to paraphrase supports both writing ability and reading comprehension, particularly for less skilled learners. As such, educational tools that integrate automated evaluations of paraphrases can be used to provide timely feedback to enhance learner paraphrasing skills more efficiently and effectively. Paraphrase identification is a popular NLP classification task that involves establishing whether two sentences share a similar meaning. Paraphrase quality assessment is a slightly more complex task, in which pairs of sentences are evaluated in-depth across multiple dimensions. In this study, we focus on four dimensions: lexical, syntactical, semantic, and overall quality. Our study introduces and evaluates various machine learning models using handcrafted features combined with Extra Trees, Siamese neural networks using BiLSTM RNNs, and pretrained BERT-based models, together with transfer learning from a larger general paraphrase corpus, to estimate the quality of paraphrases across the four dimensions. Two datasets are considered for the tasks involving paraphrase quality: ULPC (User Language Paraphrase Corpus) containing 1998 paraphrases and a smaller dataset with 115 paraphrases based on children’s inputs. The paraphrase identification dataset used for the transfer learning task is the MSRP dataset (Microsoft Research Paraphrase Corpus) containing 5801 paraphrases. On the ULPC dataset, our BERT model improves upon the previous baseline by at least 0.1 in F1-score across the four dimensions. When using fine-tuning from ULPC for the children dataset, both the BERT and Siamese neural network models improve upon their original scores by at least 0.11 F1-score. The results of these experiments suggest that transfer learning using generic paraphrase identification datasets can be successful, while at the same time obtaining comparable results in fewer epochs.

Literacy skills are critical for future success, yet over 60% of high school seniors lack proficient reading skills according to standardized tests. The focus on high stakes, standardized test performance may lead educators to “teach-to-the-test” rather than supporting transferable comprehension strategies that students need. StairStepper can fill this gap by blending necessary test prep and reading comprehension strategy practice in a fun, game-based environment. StairStepper is an adaptive literacy skill training game within Interactive Strategy Training for Active Reading and Thinking (iSTART) intelligent tutoring system. StairStepper is unique in that it models text passages and multiple-choice questions of high-stakes assessments, iteratively supporting skill acquisition through self-explanation prompts and scaffolded, adaptive feedback based on performance and self-explanations. This paper describes an experimental study employing a delayed-treatment control design to evaluate users’ perceptions of the StairStepper game and its influence on reading comprehension scores. Results indicate that participants enjoyed the visual aspects of the game environment, wanted to perform well, and considered the game feedback helpful. Reading comprehension scores of students in the treatment condition did not increase. However, the comprehension scores of the control group decreased. Collectively, these results indicate that the StairStepper game may fill the intended gap in instruction by providing enjoyable practice of essential reading comprehension skills and test preparation, potentially increasing students’ practice persistence while decreasing teacher workload.

This study examines links between essay quality and text elaboration and text cohesion. For this study, 35 students wrote two essays (on two different prompts) and for each, were given 15 minutes to elaborate on their original text. An expert in discourse comprehension then modified the original and elaborated essays to increase cohesion, resulting in a 2 (prompt) x 2 (original content, elaborated content) x 2 (original cohesion, improved cohesion) design. Expert raters scored the essays for overall quality and text coherence. In terms of overall essay quality, increasing text content (i.e., elaboration) and improving cohesion both led to significant gains in expert judgments of writing quality, and a combination of both elaboration and improved cohesion led to increased scores over increased cohesion alone. Judgments of text coherence were increased by improved cohesion (but not elaboration); and a combination of both elaboration and improved cohesion led to higher human ratings of coherence in comparison to the original and elaborated versions. The results have important implications for writing theories, writing success, writing pedagogy, and standardized testing.

Readers often struggle to identify the main ideas in expository texts. Existing research and instruction provide some guidance on how to encourage readers to identify main ideas. However, there is substantial variability in how main ideas are operationalized and how readers are prompted to identify main ideas. This variability hinders identification of best practices for instruction and intervention. The goal of the current series of experiments was to systematically examine the extent to which different tasks (e.g., selecting main ideas vs. deleting details) and different operationalizations of main ideas (e.g., "important ideas" vs. "main ideas") influenced adult readers' identification of sentences containing main ideas as they read 11 expository texts. Across experiments, the results showed that readers were generally unreliable in identifying main idea sentences; however, they were more reliable when they were instructed to select main idea sentences compared to when they were instructed to delete sentences comprised of details, and more skilled readers were more reliable than less skilled readers. The findings from the current experiments challenge existing instructional approaches and call for additional research to better understand readers' main idea selection.

Modeling reading comprehension processes is a critical task for Learning Analytics, as accurate models of the reading process can be used to match students to texts, identify appropriate interventions, and predict learning outcomes. This paper introduces an improved version of the Automated Model of Comprehension, namely version 4.0. AMoC has its roots in two theoretical models of the comprehension process (i.e., the Construction-Integration model and the Landscape model), and the new version leverages state-of-the-art Large Language models, more specifically ChatGPT, to have a better contextualization of the text and a simplified construction of the underlying graph model. Besides showcasing the usage of the model, the study introduces three in-depth psychological validations that argue for the model's adequacy in modeling reading comprehension. In these studies, we demonstrated that AMoC is in line with the theoretical background proposed by the Construction-Integration and Landscape models, and it is better at replicating results from previous human psychological experiments than its predecessor. Thus, AMoC v4.0 can be further used as an educational tool to, for example, help teachers design better learning materials personalized for student profiles. Additionally, we release the code from AMoC v4.0 as open source in a Google Collab Notebook and a GitHub repository.

Readers build a mental representation of text during reading. The coherence building processes readers use to build a mental representation during reading is key to comprehension. We examined the effects of self- explanation on coherence building processes as undergraduates (n =51) read five complementary texts about natural selection and completed a post-reading measure of topic knowledge. Participants generated constructed responses (verbal protocols) while reading. We varied the use of constructed response prompt (self-explain vs. think-aloud) and constructed response format (typed vs. oral) to examine their impact on the quality of readers' constructed responses and their coherence building processes (i.e., cohesion). Participants who received self- explanation instructions had higher quality constructed responses than participants who received think-aloud instructions, regardless of whether responses were typed or spoken aloud. Natural Language Processing (NLP) analyses indicated that participants who were prompted to self-explain generated more cohesive responses than those who were prompted to think-aloud. Participants who received self-explanation instructions had more coherent mental models during reading and typing was related to more cohesive responses when participants were asked to self-explain. Participants whose constructed responses were more lexically cohesive during reading had better performance on the post-reading test.

The authors review five major findings in reading comprehension and their implications for educational practice. First, research suggests that comprehension skills are separable from decodingprocesses and important at early ages, suggesting that comprehension skills should be targeted early, even before the child learns to read. Second, there is an important distinction between readingprocesses and products, as well as their causal relationship: processes lead to certain products. Hence, instructional approaches and strategies focusing on processes are needed to improve students’reading performance (i.e., product). Third, inferences are a crucial component of skilled comprehension. Hence, children need scaffolding and remediation to learn to generate inferences, even when they know little about the text topic. Fourth, comprehension depends on a complex interaction between the reader, the characteristics of the text, and the instructional task, highlighting the need for careful selection of instructional materials for individual students and specific groups of students. Finally, educators may benefit from heightened awareness of the limitations and inadequacies of standardized reading comprehension assessments, as well as the multidimensionality of comprehension to better understand their students’ particular strengths and weaknesses.

We examined young readers’ comprehension as a function of text genre (narrative, science), text cohesion (high, low), and readers’ abilities (reading decoding skills and world knowledge). The overarching purpose of this study was to contribute to our understanding of the fourth grade slump. Children in grade 4 read four texts, including one high and one low cohesion text from each genre. Comprehension of each text was assessed with 12 multiple-choice questions and free and cued recall. Comprehension was enhanced by increased knowledge: high knowledge readers showed bettercomprehension than low knowledge readers and narratives were comprehended better than science texts. Interactions between readers’ knowledge levels and text characteristics indicated that thechildren showed larger effects of knowledge for science than for narrative texts, and those with more knowledge better understood the low cohesion, narrative texts, showing a reverse cohesion effect.Decoding skill benefited comprehension, but effects of text genre and cohesion depended less on decoding skill than prior knowledge. Overall, the study indicates that the fourth grade slump is at leastpartially attributable to the emergence of complex dependencies between the nature of the text and the reader’s prior knowledge. The results also suggested that simply adding cohesion cues, and notexplanatory information, is not likely to be sufficient for young readers as an approach to improving comprehension of challenging texts.

The exponential growth of scientific publications increases the effort required to identify relevant articles. Moreover, the scale of studies is a frequent barrier to research as the majority of studies are low or medium-scaled and do not generalize well while lacking statistical power. As such, we introduce an automated method that supports the identification of large-scale studies in terms of population. First, we introduce a training corpus of 1229 manually annotated paragraphs extracted from 20 articles with different structures and considered populations. Our method considers prompting a FLAN-T5 language model with targeted questions and paragraphs from the previous corpus so that the model returns the number of participants from the study. We adopt a dialogic extensible approach in which the model is asked a sequence of questions that are gradual in terms of focus. Second, we use a validation corpus with 200 articles labeled for having N larger than 1000 to assess the performance of our language model. Our model, without any preliminary filtering with heuristics, achieves an F1 score of 0.52, surpassing previous analyses performed that obtained an F1 score of 0.51. Moreover, we achieved an F1 score of 0.69 when combined with previous extraction heuristics, thus arguing for the robustness and extensibility of our approach. Finally, we apply our model to a newly introduced dataset of ERIC publications to observe trends across the years in the Education domain. A spike was observed in 2019, followed by a decrease in 2020 and, afterward, a positive trend; nevertheless, the overall percentage is lower than 3%, suggesting a major problem in terms of scale and the need for a change in perspective.

This interdisciplinary study comprises two complementary analyses on a corpus of journal abstracts written in English by American, British, and Japanese scientists. The fi rst analysis uses the computational tool Coh-Metrix to assess text at the discourse level. The second analysis uses the computational tool the Gramulator to compare the frequency of n-grams across the three sources of abstracts. The Coh-Metrix and Gramulator analyses both suggest signifi cant differences between all three varieties of English. The greatest differences were apparent when comparing abstracts written by Japanese and English speakers; however, a number of differences were also apparent when comparing the British English and American English varieties. The results lend weight to the conclusion that native-English speakers (reviewer, editor, or reader) of either the British or American variety may interpret Japanese-English texts as lacking in key areas of the proto-typical style of the English register. Our findings provide information for instructors, course developers, and scientists on how and where text might be modifi ed in order to facilitate the production of more native-English-like representations.