Search

The extent to which learners benefit from instruction may be largely dependent on their individual abilities. However, there is relatively little work on the interaction between instructional effectiveness in second language learning and learner individual factors. In this study, we investigated the relationship between instruction, individual differences in cognitive abilities (working memory and declarative memory), and second language vocabulary acquisition in the context of web-based intelligent computer assisted language learning (ICALL). To this end, 127 adult learners of English, predominantly advanced-level, German-speaking learners, read news texts on the web for about two weeks using an ICALL system under two instructional conditions: form-focused and meaning-focused instruction. Learners in the form-focused condition read and completed automatically-generated multiple-choice gaps where phrasal verbs appeared in the text, while learners in the meaning-focused condition simply read and did not complete any gaps. Mixed-effects regression analyses showed that working memory was associated with vocabulary acquisition and that this association depended on the instructional context, with working memory being predictive of learning only in the form-focused condition, suggesting an aptitude-treatment interaction. Furthermore, declarative memory abilities were related to learning only as measured by the Continuous Visual Memory Task, and the relationship was not moderated by instructional condition. Overall, the study contributes to accounting for variability in second language learning in general, as well as in instructional contexts supported by intelligent CALL.

The paper tackles a central question in the field of Intelligent Computer-Assisted Language Learning (ICALL): How can language learning tasks be conceptualized and made explicit in a way that supports the pedagogical goals of current Foreign Language Teaching and Learning and at the same time provides an explicit characterization of the Natural Language Processing (NLP) requirements for providing feedback to learners completing those tasks? We argue that the successful implementation of language learning tasks that can be automatically assessed demands a design process in which both pedagogical and computational requirements feed each other. Extending well-established work in Task-Based Instruction (TBI) and Language Testing, we propose a framework that helps us (a) elucidate the formal features of foreign language learning activities, (b) characterize the relation between the expected and the actually elicited learner language, and (c) assess how the variability in the learner responses impacts interpretation and computational techniques for the automatic analysis of learner language. To validate our approach we apply the framework to two writing tasks for learners of English as a foreign language. Our analysis highlights the relevance of spelling out the pedagogical and linguistic goals of language learning tasks in order to successfully characterize the language expected and actually elicited in learner responses, to evaluate the realization of the task, and to support the design of effective NLP strategies. Given the combination of design- and data-driven perspectives, the framework supports an iterative approach to the creation of language learning tasks and ICALL materials.

How can state-of-the-art computational linguistic technology reduce the workload and increase the efficiency of language teachers? To address this question, we combine insights from research in second language acquisition and computational linguistics to automatically generate text-based questions to a given text. The questions are designed to draw the learner's attention to target linguistic forms -- phrasal verbs, in this particular case -- by requiring them to use the forms or their paraphrases in the answer. Such questions help learners create form-meaning connections and are well suited for both practice and testing. We discuss the generation of a novel type of question combining a "wh"- question with a gapped sentence, and report the results of two crowdsourcing evaluation studies investigating how well automatically generated questions compare to those written by a language teacher. The first study compares our system output to gold standard human-written questions via crowdsourcing rating. An equivalence test shows that automatically generated questions are comparable to human-written ones. The second crowdsourcing study investigates two types of questions ("wh"- questions with and without a gapped sentence), their perceived quality, and the responses they elicit. Finally, we discuss the challenges and limitations of creating and evaluating question-generation systems for language learners.

Silent reading is the primary mode of reading for proficient readers, and therefore, silent reading fluency is often assessed in research and practice. However, little is known about the validity of the tests administered to students with different language backgrounds. Given that academic language is assumed to be especially challenging for students with a non-German home language, one might wonder if effects of academic language proficiencies can be found on these tests. In the present study, we explored whether, owing to academic language demands, students with a non-German home language (N = 748) would be found to be at a greater disadvantage than their monolingual-home peers (N = 1669) on the most frequently used silent reading fluency test in Germany. Using differential item functioning (DIF) analyses, we found specific item difficulties to the disadvantage of the students with a non-German home language. This DIF was linked to the academic language features of the sentences.

How can we identify authentic reading material that matches the learner's proficiency and fosters their language development? Traditionally, this involves assigning a one-dimensional label to the text that identifies the grade or proficiency level of the learners that the text is intended for. Such an approach is inadequate given that both the text complexity and proficiency constructs are multi-dimensional in nature. We propose to instead link readers and texts through multidimensional vectors characterizing the linguistic complexity of the reading material and that of texts written by the learners as proxy of their proficiency level. We first validate the approach using a leveled reading corpus by showing that vector distances computed on the complexity representations can serve the function of the traditional labels. We then highlight the advantage of the multi-dimensional approach using data from a continuation writing task, showing that it makes it possible to study individual complexity dimensions and to explore different degrees of challenge for different dimensions. Our approach essentially makes it possible to empirically investigate the +1 of Krashen's i+1, the challenge that best fosters development given the learner's interlanguage. On the practical side, we discuss an ICALL system demonstrating the viabilityof the approach in real-life..

The linguistic characteristics of text productions depend on various factors, including individual language proficiency as well as the tasks used to elicit the production. To date, little attention has been paid to whether some writing tasks are more suitable than others to represent and differentiate students' proficiency levels. This issue is especially relevant in the context of high‐stakes language examinations. In this study, we investigated task effects in English as a foreign language (EFL) Abitur examinations, the high‐stakes test qualifying for higher education admission in Germany. Based on texts produced by 362 students, we examined (a) if and how student writings differ in their linguistic complexity and (b) if the EFL Abitur tasks are equally suited to differentiate the students' proficiency levels. We used a broad operationalization of linguistic complexity, including measures from various linguistic domains in the computational analysis of the texts. The results of our mixed‐effects models show that student texts differ in their linguistic complexity primarily by the functional needs of the task types employed (summary, analysis, and argumentation). Furthermore, writing tasks that demand high independence in language performance are best suited to differentiate between proficiency levels, which becomes especially evident in vocabulary choice and usage. [ABSTRACT FROM AUTHOR]

Search engines have been a major factor in making the web the successful and widely used information source it is today. Generally speaking, they make it possible to retrieve web pages on a topic specified by the keywords entered by the user. Yet web searching currently does not take into account which of the search results are comprehensible for a given user--an issue of particular relevance when considering students in an educational setting. And current search engines do not support teachers in searching for language properties relevant for selecting texts appropriate for language students at different stages in the second language acquisition process. At the same time, raising language awareness is a major focus in second language acquisition research and foreign language teaching practice, and research since the 20s has tried to identify indicators predicting which texts are comprehensible for readers at a particular level of ability. For example, the military has been interested in ensuring that workers at a given level of education can understand the manuals they need to read in order to perform their job. We present a new search engine approach which makes it possible for teachers to search for texts both in terms of contents and in terms of their reading difficulty and other language properties. The implemented prototype builds on state-of-the art information retrieval technology and exemplifies how a range of readability measures can be integrated in a modular fashion.

Assessment of text readability is important for assigning texts at the appropriate level to readers at different proficiency levels. The present research approached readability assessment from the lexical perspective of word frequencies derived from corpora assumed to reflect typical language experience. Three studies were conducted to test how the word-level feature of word frequency can be aggregated to characterise text-level readability. The results show that an effective use of word frequency for text readability assessment should take a range of characteristics of the distribution of words frequencies into account. For characterizing text readability, taking into account the standard deviation in addition to the mean word frequencies already significantly increases results. The best results are obtained using the mean frequencies of the words in language frequency bands or in bands obtained by agglomerative clustering of the word frequencies in the documents--though a comparison of within-corpus and cross-corpus results shows the limited generalizability of using high numbers of fine-grained frequency bands. Overall, the study advances our understanding of the relationship between word frequency and text readability and provides concrete options for more effectively making use of lexical frequency information in practice.

An adequate level of linguistic complexity in learning materials is believed to be of crucial importance for learning. The implication for school textbooks is that reading complexity should differ systematically between grade levels and between higher and lower tracks in line with what can be called the systematic complexification assumption. However, research has yet to test this hypothesis with a real-world sample of textbooks. In the present study, we used automatic measures from computational linguistic research to analyze 2,928 texts from geography textbooks from four publishers in Germany in terms of their reading demands. We measured a wide range of lexical, syntactic, morphological, and cohesion-related features and developed text classification models for predicting the grade level (Grades 5 to 10) and school track (academic vs. vocational) of the texts using these features. We also tested ten linguistic features that are considered to be particularly important for a reader's understanding. The results provided only partial support for systematic complexification. The text classification models showed accuracy rates that were clearly above chance but with considerable room for improvement. Furthermore, there were significant differences across grade levels and school tracks for some of the ten linguistic features. Finally, there were marked differences among publishers. The discussion outlines key components for a systematic research program on the causes and consequences of the lack of systematic complexification in reading materials.

This article discusses two types of opportunities for interdisciplinary collaboration between computational linguistics (CL) and language learning research. We target the connection between data and theory in second language (L2) research and highlight opportunities to (a) enrich the options for obtaining data and (b) support the identification and valid interpretation of relevant learner data. We first characterize options, limitations, and potential for obtaining rich data on learning: from Web-based intervention studies supporting the collection of experimentally controlled data to online workbooks facilitating large-scale, longitudinal corpus collection for a range of learning tasks and proficiency levels. We then turn to the question of how corpus data can systematically be used for L2 research, focusing on the central role that linguistic corpus annotation plays in that regard. We show that learner language poses particular challenges to human and CL analysis and requires more interdisciplinary discussion of analysis frameworks and advances in annotation schemes.

Despite the promise of research conducted at the intersection of computer-assisted language learning (CALL), natural language processing, and second language acquisition, few studies have explored the potential benefits of using intelligent CALL systems to deepen our understanding of the process and products of second language (L2) learning. The strategic use of technology offers researchers novel methodological opportunities to examine how incremental changes in L2 development occur during treatment as well as how the longitudinal impacts of experimental interventions on L2 learning outcomes occur on a case-by-case basis. Drawing on the pilot results from a project examining the effects of automatic input enhancement on L2 learners' development, this article explores how the use of technology offers additional methodological and analytical choices for the investigation of the process and outcomes of L2 development, illustrating the opportunities to study what learners do "during" visually enhanced instructional activities.

Large-scale learner corpora collected from online language learning platforms, such as the EF-Cambridge Open Language Database (EFCAMDAT), provide opportunities to analyze learner data at an unprecedented scale. However, interpreting the learner language in such corpora requires a precise understanding of tasks: How does the prompt and input of a task and its functional requirements influence task-based linguistic performance? This question is vital for making large-scale task-based corpora fruitful for second language acquisition research. We explore the issue through an analysis of selected tasks in EFCAMDAT and the complexity and accuracy of the language they elicit.

Intelligent language tutoring systems (ILTS) typically analyze learner input to diagnose learner language properties and provide individualized feedback. Despite a long history of ILTS research, such systems are virtually absent from real-life foreign language teaching (FLT). Taking a step toward more closely linking ILTS research to real-life FLT, in this article we investigate the connection between FLT activity design and the system architecture of an ILT system. We argue that a demand-driven, annotation-based natural language processing (NLP) architecture is well-suited to handle the demands posed by the heterogeneous learner input which results when supporting a wider range of FLT activity types. We illustrate how the unstructured information management architecture (UIMA) can be used in an ILTS, thereby connecting the specific needs of activities in foreign language teaching to the current research and development of NLP architectures in general. Making the conceptual issues concrete, we discuss the design and realization of a UIMA-based reimplementation of the NLP in the TAGARELA system, an intelligent web-based tutoring system supporting the teaching and learning of Portuguese. (Contains 1 note, 2 tables, and 4 figures.)

This paper explores the motivation and prerequisites for successful integration of Intelligent Computer-Assisted Language Learning (ICALL) tools into current foreign language teaching and learning (FLTL) practice. We focus on two aspects, which we argue to be important for effective ICALL system development and use: (i) the relationship between activity design and restrictions needed to make natural language processing tractable and reliable, and (ii) pedagogical considerations and the influence of activity design choices on the integration of ICALL systems into FLTL practice.