Search

Building on previous studies investigating the multidimensional nature of lexical use in task-based L2 performance, this study clarified the roles that the distinct lexical features play in predicting vocabulary proficiency in a corpus of L2 Oral Proficiency Interviews (OPI). A total of 85 OPI samples were rated by three separate raters based on a Common European Frame of Reference (CEFR) based rubric in terms of their linguistic range. The interview transcription was analyzed for 56 lexical and phraseological indices using modern natural language processing tools. The result of an exploratory factor analysis (EFA) revealed that the 56 indices tapped into 10 distinct factors of lexical use in OPI: three factors related to content words, three related to n-grams, three lexical collocation factors, and one function-word factor. A subsequent Bayesian mixed-effect ordinal regression indicated that six out of the 10 factors meaningfully predicted the CEFR levels on Range with reasonable accuracy (quadratic kappa coefficient = .81 with the human rating). The result highlights the distinct roles that multiple content-word, collocation, and function-word factors play in characterizing the linguistic range in a CEFR-based assessment of OPI. The implication for the assessment of lexical richness, as well as future directions of this research domain, are discussed.

Responding to the increasing need for automated writing evaluation (AWE) systems to assess language use beyond lexis and grammar (Burstein et al., 2016), we introduce a new approach to identify rhetorical features of stance in academic English writing. Drawing on the discourse-analytic framework of engagement in the Appraisal analysis (Martin & White, 2005), we manually annotated 4,688 sentences (126,411 tokens) for eight rhetorical stance categories (e.g., PROCLAIM, ATTRIBUTION) and additional discourse elements. We then report an experiment to train machine learning models to identify and categorize the spans of these stance expressions. The best-performing model (RoBERTa + LSTM) achieved macro-averaged F1 of .7208 in the span identification of stance-taking expressions, slightly outperforming the intercoder reliability estimates before adjudication (F1 = .6629)., The 18th Workshop on Innovative Use of NLP for Building Educational Applications

Supplemental material, sj-docx-1-ltr-10.1177_13621688221122138 for The contribution of guessing from context and dictionary use to receptive and productive vocabulary knowledge: A structural equation modeling approach by Takumi Uchihara, Masaki Eguchi and Jon Clenton in Language Teaching Research

Supplemental material, sj-docx-1-ltr-10.1177_13621688221122138 for The contribution of guessing from context and dictionary use to receptive and productive vocabulary knowledge: A structural equation modeling approach by Takumi Uchihara, Masaki Eguchi and Jon Clenton in Language Teaching Research

Supplemental material, sj-docx-1-las-10.1177_00238309211013865 for To What Extent is Collocation Knowledge Associated with Oral Proficiency? A Corpus-Based Approach to Word Association by Takumi Uchihara, Masaki Eguchi, Jon Clenton, Kristopher Kyle and Kazuya Saito in Language and Speech

Supplemental material, sj-docx-1-las-10.1177_00238309211013865 for To What Extent is Collocation Knowledge Associated with Oral Proficiency? A Corpus-Based Approach to Word Association by Takumi Uchihara, Masaki Eguchi, Jon Clenton, Kristopher Kyle and Kazuya Saito in Language and Speech