Search

Few studies have yet described concrete efforts by researchers in applied linguistics to systematically impact language policy. In linguistics, there is a general lack of published work on interactions between research and policy, and authors have decried a general dearth of policy literacy among applied linguists. The goal of the current paper is to describe how applied linguistics research can bring about policy impact by presenting a narrative account of one approach aimed at impacting university admission language policies. The first part of the paper presents research-based recommendations regarding language requirements for international students, the second focuses on mechanisms to communicate these recommendations to policy makers. The case study presented in this paper serves to argue that applied linguistic research and policy impact can go hand in hand, on the condition that policy recommendations are concrete, timely, seen as relevant, and researchers have a fundamental understanding of the policy-making context.

The Peabody Picture Vocabulary Test (PPVT) is a widely used test of receptive vocabulary, but no researchers to date have examined the performance of low-educated, low-literate L2 adults, or compared these individuals' performances to their more highly educated peers. In this study, we used many-facet Rasch analysis and mixed-effects linear regression to determine the impact of educational background and other demographic variables on PPVT test performance. The analyses rely on the performance data of 1,014 adult learners of Dutch as a second language on the Dutch version of the PPVT (PPVT-III-NL). The results show that a substantial proportion of score variance can be attributed to educational background variables and to the educational tracks the participants followed. These tracks, which cater to the needs of different L2 learner profiles, appear to exacerbate rather than mediate any performance differences. Although this study provides evidence of performance differences and differential item functioning resulting from linguistic, demographic, and educational variables, it offers no data to invalidate the use of the PPVT on low-educated L2 adults.

Background: In order to continue educational provision during the COVID-19 pandemic, many education systems switched to some form of Emergency Remote Teaching (ERT) in 2020. Research suggests that this may have disproportionately affected students from underprivileged communities. In this context, there is, therefore, a pressing need to consider how ERT may have impacted learners who are likely to be most affected by educational inequalities, including newly arrived migrant students (NAMS). Purpose: As studies have highlighted the particular importance of interaction for effective distance learning, the research aimed to examine how NAMS in Flemish secondary schools experienced interaction with learning content, teachers and fellow students in online classes during ERT. Methods: A total of six semi-structured focus group interviews were conducted with 23 NAMS from six secondary schools in Belgium. The interviews were centred on questions about participants' experiences with online interaction and participation. All focus groups were audio recorded and transcribed verbatim, after which transcriptions were coded and analysed qualitatively. Findings: The analysis revealed that, during ERT, participants reported experiencing a higher amount of learner-content interaction, mainly consisting of self-study material and tasks. Participants also indicated a lack of learner-teacher interaction, which was attributed to the shortage of speaking opportunities and students' decreased inclination to interact with instructors. In addition, most participants reportedly experienced few opportunities for learner-learner interaction during online classes. Conclusions: Study findings provided insight into the NAMS' experiences of being suddenly and unexpectedly compelled to be autonomous learners during ERT. Students' reports suggest that the quality of learner-content interaction may have been compromised by limited learner--teacher interaction in the ERT situation. This draws attention more generally to the importance of interactive learning in the support of NAMS and the need for educators to be empowered to develop interactivity-rich remote learning environments.

Rating scale development in the field of language assessment is often considered in dichotomous ways: It is assumed to be guided either by expert intuition or by drawing on performance data. Even though quite a few authors have argued that rating scale development is rarely so easily classifiable, this dyadic view has dominated language testing research for over a decade. In this paper we refine the dominant model of rating scale development by drawing on a corpus of 36 studies identified in a systematic review. We present a model showing the different sources of scale construct in the corpus. In the discussion, we argue that rating scale designers, just like test developers more broadly, need to start by determining the purpose of the test, the relevant policies that guide test development and score use, and the intended score use when considering the design choices available to them. These include considering the impact of such sources on the generalizability of the scores, the precision of the post-test predictions that can be made about test takers' future performances and scoring reliability. The most important contributions of the model are that it gives rating scale developers a framework to consider prior to starting scale development and validation activities.

This paper constitutes an edited transcript of two online panels, conducted with four scholars whose complementary expertise regarding print literacy and migration offers a thought-provoking and innovative window on the representation of print literacy in applied linguistic research and in migration policy. The panel members are experts on language policy, literacy, proficiency and human capital research. Together, they address a range of interrelated matters: the constructs of language proficiency and literacy (with significant implication for assessment), the idea of literacy as human capital or as a human right, the urgent need for policy literacy among applied linguists, and the responsibility of applied linguistics in the literacy debate.

To date, language testing research has devoted little attention to adult L2 learners with low levels of alphabetic print literacy (LESLLA), even though this population makes up for a substantial proportion of the candidature of language tests used for migration purposes. This special issue focuses on LESLLA learners, shows how literacy impacts test performance beyond what might be considered construct relevant, and how a focus on LESLLA learners invites a reconceptualization of commonly held notions of literacy, test misuse and even validation.

Elicited imitation (EI) is a much-used measurement instrument in applied linguistics, and it is considered a reliable and quick assessment of holistic speaking ability and implicit grammar knowledge. To date, however, EI research has overwhelmingly relied on highly educated participants. Only a few small-scale EI studies in applied linguistics have considered low-literate learners. Using Item Response Theory (IRT) and inferential parametric and nonparametric statistics, this study examined the EI performance of 113 L2 learners of Dutch with diverging educational backgrounds. All participants were enrolled in A1 and A2 Dutch L2 courses. Additionally, this study examined to what extent EI performances align with standardized tests of speaking and receptive vocabulary. The results indicate that EI can be used with both highly literate and low-educated participants, who will--however--be outperformed by higher educated learners. Especially the repetition of sentences containing pseudowords appears to result in substantial performance differences between the two groups.

Introduction: Previous research has uncovered various inequalities for immigrant students in education systems worldwide. Yet, the degree to which diverse background factors, such as socio-economic status and prior educational trajectory, contribute to these educational disadvantages remains uncertain. Methods: Using data from the Flemish administrative database (2009-2019) on 48,340 mainstream students and 1,022 Newly Arrived Migrant Students (NAMS) across 685 schools, this study examines NAMS' passing and dropout rates in Flemish secondary education in comparison to the outcomes of their peers without a recentmigration background. Data included demographic background variables, school variables, and indicators of educational outcomes. Results: A cross-classified multilevel regression analysis showed that, after controlling for background variables, NAMS still show higher probabilities of failing a year and of dropping out. Moreover, the impact of SES is stronger for native students than for NAMS. Discussion: These results suggest the existence of a migrant background effect disadvantaging NAMS in Flemish secondary education. [ABSTRACT FROM AUTHOR]

Recently, research in second language acquisition has seen an increased attention on low-educated low-literate (LESLLA) learners. However, few of the existing instruments to measure L2 proficiency have been validated for use with this population. In this paper, we examine how adult L2 learners with diverging educational backgrounds perform on and engage with a L2 receptive skills test that was specifically developed with LESLLA learners in mind. To gauge test perception and strategies, 48 less- and highly-educated L2 learners participated in cognitive interviews. Additionally, a live test was administered to 616 adult L2 learners to assess the impact of educational background on L2 performance. Combining psychometric and interview data, this study not only provides initial validation of the test for a diverse group of adult L2 learners, but also offers insights into the test-taking experiences of LESLLA learners, addressing a long-neglected group in applied linguistics research. [ABSTRACT FROM AUTHOR]

University entrance language tests are often administered under the assumption that even if language proficiency does not determine academic success, a certain proficiency level is still required. Nevertheless, little research has focused on how well L2 students cope with the linguistic demands of their studies in the first months after passing an entrance test. Even fewer studies have taken a longitudinal perspective. Set in Flanders, Belgium, this study examines the opinions and experiences of 24 university staff members and 31 international L2 students, of whom 20 were tracked longitudinally. Attention is also given to test/retest results, academic score sheets, and class recordings. To investigate the validity of inferences made on the basis of L2 students' scores, Kane's (2013) Interpretation/Use Argument approach is adopted, and principles from political philosophy are applied to investigate whether a policy that discriminates among students based on language test results can be considered just. It is concluded that the receptive language requirements of university studies exceed the expected B2 level and that the Flemish entrance tests include language tasks that are of little importance for first-year students. Furthermore, some of the students who failed the entrance test actually managed quite well in their studies -- a result that entails broad implications concerning validation and justice even outside the study's localized setting.

In Flanders, Belgium, university admission of undergraduate international L2 students requires a certificate of an accredited test of Dutch. The two main university entrance tests used for certification share highly comparable oral components and CEFR-based oral rating criteria. This article discusses to what extent ratings on the oral components of these tests can be compared. The data used are the ratings of the oral performances of the same 82 candidates on both oral test components, which were administered within the same week. The correlation on the overall scores is high, but lower on the oral test component. Further analyses, including linear regression and multifaceted Rasch analysis, indicate that the B2 level was interpreted differently in the two tests. The results show that using the same language proficiency scales as the basis for rating scale criteria may lead to superficial correspondences or a perceived equivalence but does not necessarily lead to greater comparability of shared criteria. The findings of this study are especially useful for contexts in which different tests use similar criteria that are based on the same descriptors, and comparability is only assumed.

Fifteen years after its publication, the Common European Framework of Reference for Languages is a commonly used document in language tests and policies across Europe. This article considers the CEFR's impact on university entrance language tests and policies that are used to regulate the entrance of international L2 students who wish to study in a national language of the host country. Using a qualitative approach, this study aims to (a) outline and compare the target language demands toward L2 students entering European universities and to (b) determine the impact of the CEFR on European university entrance policies, tests, and testers. This article offers an overview of the university entrance language requirements for foreign L2 students in 28 European countries or regions with an autonomous educational policy. It is based on structured interviews with 30 respondents involved with university entrance test development and knowledgeable about university entrance policies in their context. The results show that the CEFR is omnipresent in European university entrance language tests and that the B2 is the most commonly used level in that context. The data also show that normative CEFR use is very common and that in many contexts CEFR levels are misused for marketing purposes or to control university admission.

In the context of policy-driven language testing for citizenship, a growing body of research examines the political justifications and ethical implications of language requirements and test use. However, virtually no studies have looked at the role that language testers play in the evolution of language requirements. Critical gaps remain in our understanding of language testers' first-hand experiences interacting with policymakers and how they perceive the use of tests in public policy. We examined these questions using an exploratory design and semi-structured interviews with 28 test executives representing 25 exam boards in 20 European countries. The interviews were transcribed and double coded in NVivo (weighted kappa =.83) using a priori and inductive coding. We used a horizontal analysis to evaluate responses by participant and a vertical analysis to identify between-case themes. Findings indicate that language testers may benefit from policy literacy to form part of policy webs wherein they can influence instrumental decisions concerning language in migration policy. [ABSTRACT FROM AUTHOR]

A growing scholarly literature in a subfield of applied linguistics focuses on language testing and language requirements for migrants. We sought to understand what theoretical paradigms and methodological approaches have framed this research and propose future research directions. To this end, we conducted a systematic review of articles on language testing and migration published between 2001 and 2022 in the applied linguistics field. For each of the 46 studies which met inclusion/exclusion criteria, we randomly selected a control article from the same journals. Both the test corpus and the control corpus were double coded for theoretical and methodological characteristics and analysed using contingency tables, Mann–Whitney tests, and forward stepwise logistic regression to predict group membership. The findings revealed that literature on language testing and migration is highly reliant on critical theory and less reliant on features associated with empirical methods. We suggest that research in this domain would benefit from methodological reflexivity (i.e. questioning prevalent assumptions and employing theoretical variety) and that complementary approaches could lead to a more nuanced research consensus and promote research-policy collaboration. [ABSTRACT FROM AUTHOR]

Considering scoring validity as encompassing both reliable rating scale use and valid descriptor interpretation, this study reports on the validation of a CEFR-based scale that was co-constructed and used by novice raters. The research questions this paper wishes to answer are (a) whether it is possible to construct a CEFR-based rating scale with novice raters that yields reliable ratings and (b) allows for a uniform interpretation of the descriptors. Additionally, this study focuses on the question whether co-constructing a rating scale with novice raters helps to stimulate a shared interpretation of the descriptors over time. For this study, six novice raters employed a CEFR-based scale that had been co-constructed by themselves and 14 peers to rate 200 spoken and written performances in a missing data design. The quantitative data were analysed using item response theory, classical test theory and principal component analysis. The focus group data, collected after the rating process, were transcribed and coded using both a priori and inductive coding. The results indicate that novice raters can reliably use the CEFR-based rating scale, but that the interpretations of the descriptors, in spite of training and co-construction, are not as homogeneous as the inter-rater reliability would suggest.

In social interaction, foreign language accent and comprehensibility impact how we perceive our conversational partners. In recent years, research interest in these constructs has been on the rise, while many issues remain underexposed. These issues include the relationship between comprehension and accent on the one hand, and background variables of both learner and assessor on the other. Since most research to date has been conducted with highly educated and advanced learners of English as a second or foreign language, we do not know to what extent those results can be generalised to a wider population that includes beginning learners of Dutch as a second language from various educational backgrounds. In addition, little research has been done into the comparability of the judgements of trained and non-trained assessors. In the current study, we compared the judgments of four trained evaluators with the intuitive judgments of 272 non-trained evaluators (first-year students at various Educational Bachelor's Programmes in Primary Education). The first group of raters evaluated the speaking performance of 116 learners of Dutch as a second language using standardised criteria, the second group used more subjective criteria. The results show that the two groups of evaluators make very similar judgements and that these judgements are mainly related to two background variables: the nature of the NT2 course the learners followed (also an indicator of prior education and cognitive ability) and the level of language proficiency of the course. [ABSTRACT FROM AUTHOR]

The article presents the discussion on Common European Framework of Reference for Languages (CEFR—Council of Europe 2001) impacting language teaching and assessment in Europe. Topics include protecting human rights, the rule of law, and parliamentary democracy; and advancing mutual understanding between nation states, language education, communication, and multilingualism.

Research in the field of Language Assessment Literacy (LAL) shows that university admission officers and policy makers are not generally well-versed in matters of LAL. Only very few studies to date have traced why this may be the case, however, and in the field of language testing few studies to date have reported on how university admission language requirements are set. Nevertheless, because of the impact of test use on university admissions, developing such knowledge is essential to the progress of LAL as a discipline. This paper reports on a qualitative study that includes all university admission policy makers in one context (Flanders, Belgium). The analyses of the interviews show that the concerns and ideas of LAL scholars and those of university admission policy makers may differ substantially. Real-world policy is determined by pragmatism and compromise and policy makers, even at universities, may fail to consider empirical findings. Because this study shows that the view of policy makers can be quite dissimilar from the traditional approach taken in the LAL literature, the authors argue that it may be as beneficial to encourage policy literacy among language testing professionals, as to expect LAL from policy makers. [ABSTRACT FROM AUTHOR]

As higher education has grown into a global enterprise, international students have become an integral part of the student population at many universities. Given this reality, it is striking that there are considerable gaps in our knowledge of whether and why international students make language gains, or fail to do so. In order to address these voids in research, this study employed a longitudinal mixed-methods design to measure oral and written gains made by international L2-Dutch students studying in universities in Flanders over an eight-month period. It also sheds light on the institutional and social contexts of the participants, as measured by longitudinal in-depth interviews. The quantitative data indicate that the participants made no demonstrable gains except on one sole indicator of written fluency, and the qualitative findings reveal that linguistic readiness, perceived linguistic inferiority, and a lack of access to the L1 community hampered students’ opportunities and willingness to engage in meaningful interaction. This study offers no immediate reason to presume that rich input alone yields language gains, even with advanced learners in an academic context. Rather, it reaffirms the hypothesis that language learning in a naturalistic setting is driven by meaningful interaction and a sense of belonging. [ABSTRACT FROM AUTHOR]

Justice has been the topic of comparatively few papers in the fields of applied linguistics or language assessment. This may be due to the lack of a clear and agreed-upon definition on the one hand, or to the difficulty of operationalizing justice for test development on the other. This paper aims to remedy both problems by discussing prior conceptualizations of justice and by introducing six justice principles, which are based on theories of distributive justice that focus on human rights, fairness, equal opportunity, and dignity. The overarching aim of this paper is to advance the debate on justice, and to provide a consistent way of considering ethical and moral dilemmas that language testers face today. [ABSTRACT FROM AUTHOR]

An introduction is presented in which the editor discusses articles in the issue on topics including "The Common European Framework of Reference (CEFR)" project; European education; and role of phraseology and language testing in CEFR scale.