Search

Writing assessment is a crucial practice in the success of teaching and learning a specific genre. The relevant literature highlights that assessing the written literacy levels of students should be well-designed in order to meet their learning needs and achieve clearly articulated goals. It also points out that there are several unaddressed issues in the existing approaches to writing assessment. Accordingly, this review paper aims to discuss some of those issues, in particular, those around the functional language analysis method, which rests upon Systemic Functional Linguistics (SFL). One of the concerns in such an approach is what the potential limitations might be when using only purely quantitative analysis, which simply calculates the frequency of a given linguistic resource in a student's writing to investigate his/her development. This review article maintains that quantitative analysis alone will offer limited insights into students' written literacy development. Therefore, it suggests employing a linguistically-observant method based on concrete stylistic evidence that considers both quantitative analyses and qualitative observations for judging written literacy levels. It concludes that this combination will prove effective for offering useful insights into writing assessment and making well-founded conclusions on how to track and gauge written literacy development. [ABSTRACT FROM AUTHOR]

Reflection is a staple of contemporary writing pedagogy and writing assessment. Although the power of reflective writing has long been understood in writing studies, the field has not made progress on articulating how to assess the reflective work. Developed at the crossroads of research in reflection and metacognition, the Index for Metacognitive Knowledge (IMK) is designed to help writing researchers, teachers, and students articulate what is being rewarded in the assessment of reflection and to articulate the role of metacognitive knowledge in critical reflective writing. The IMK was used to code final portfolio introductions from first-year writing courses in order to analyze the distribution of the three kinds of metacognitive knowledge (declarative, procedural, and conditional) and to explore the quality and complexity of students’ metacognitive knowledge. Inter-rater reliability testing on the IMK showed that it is highly reliable; the Fleiss’ kappa was 83% (K=.834). The IMK offers researchers, teachers, and students language with which to explore the unique work of reflective writing in order to develop more metacognitively rich observations. It provides a framework to explain the evolving complexity of students’ reflective writing and to assess and describe the impacts of other pedagogical interventions.

This article examines American high school students’ (N=439) self-worth protection behaviors, maladaptive coping mechanisms, and academic performance under a contract grading system, which has been understudied in contemporary secondary classrooms. The quantitative analysis revealed that under the contract grading system, 97% (n=421) earned a passing grade (i.e., A, B, or C) on the assessment and 90% (n=390) fulfilled the contract by reaching mastery (A) or proficiency (B). Compared to the previous year, students with prior experience were 19% more likely to earn an A and 16% more likely to earn a B under the grading contract despite increased workload demands. The qualitative analysis of 40 semi-structured interviews revealed that performance improved as a result of the contract’s clarity of purpose, which limited task avoidance and facilitated task-oriented effort toward a desirable goal. Students enrolled in regular courses experienced the most significant grade improvement due to clear expectations that helped them place their effort on the right tasks. The findings of this study lead to a call to action for teachers to implement contract grading in high school classrooms to clarify work expectations, improve task-oriented effort, and help students set and achieve high goals.

This study evaluated the effect of a combined feedback activity (peer feedback, computer-generated feedback and teacher feedback) on students' second language (L2) writing feedback literacy. One hundred and eighty-two Chinese university students participated in this research. Findings revealed that the intervention significantly improved students' literacy in Appreciating Feedback, Acknowledging Different Feedback Sources and Managing Affect, but not Making Judgements and Taking Action. L2 proficiency levels affected the literacy development. Low-proficiency students' feedback literacy did not change significantly. Middle-proficiency students improved significantly in Appreciating Feedback, Acknowledging Different Feedback Sources, Managing Affect, and Taking Action. High-proficiency students only improved significantly in Appreciating Feedback. Findings further reveal different degrees of difficulty for students to improve feedback literacy along its five dimensions. This study bears implications for developing students' feedback literacy in L2 writing and in other disciplinary areas, particularly regarding how teachers could use multiple feedback sources and address students' varied proficiency levels. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Technology & Chinese Language Teaching is the property of Middle Tennessee State University and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Erzincan University Journal of Education Faculty / Erzincan Üniversitesi Egitim Fakültesi Dergisi is the property of Erzincan University Faculty of Education Journal and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This editor's column provides an overview of Tamara Tate and Mark Warschauer's "Access, Digital Writing, and Achievement," Mary Stewart’s “Confronting the Ideologies of Assimilation and Colorblindness in Writing Program Assessment through Antiracist Dynamic Criteria Mapping," and Analeigh Horton's “Two Sisters and a Heuristic for Listening to Multilingual, International Students’ Directed Self-Placement Stories.”

Assessing text quality as an indication of underlying skills still remains challenging; irrespective of the approach, many studies struggle with reliability or validity problems. If writing is considered problem-solving, a report must make the reader understand the described situation and call for its mental reconstruction. Therefore, text quality may not only comprise linguistic aspects but also the cognitive-functional power of a text. The presented study aims at exploring the functionality of students' reporting texts in relation to general text-quality measures, using a corpus of accident reports written by German fifth- and ninth-graders (n = 277) prompted by a pictorial stimulus of a bike accident scenario. An online tool was developed in which 277 university students graphically reenacted the situation from one respective text according to the existence, position, and color of the involved elements. Thereafter, the match of the resulting spatial reconstructions with the original situation was assessed by two raters. While most subscales showed sufficiently high interrater reliabilities, the aggregated functionality score (α =.74) had medium-high correlations with other text-quality ratings and was comparably dependent on grade, education level, and linguistic family background. However, the correlational pattern, regression analysis, and factor analysis showed that the functionality score also contributed unique portions of variance to the assessment of writing skill that were not represented by rating measures. Moreover, the direct indication of whether a text allows for the reader's adequate cognitive representation is evident. Altogether, the approach of indicating text functionality through practical understanding offers a sound, though empirically laborious, alternative for text-quality measurement. Results are discussed with regard to the didactical strategy according to which students can improve their writing when they observe whether others can make use of their texts. [ABSTRACT FROM AUTHOR]

One aim of this article is to shift mindsets from the traditional identification of written language abilities that are “missing” in students with specific learning disabilities (SLDs) to writing assessment as the discovery of more complex language strengths foundational to success across disciplines. The multiple language-levels (MLL) perspective is grounded on identifying strengths in written language that may be present in students’ responses to classroom writing activities. We used MLL to examine the compositions of 19 students in grades 4–9 who were classified with SLDs and struggled with academic writing. Their writing assignment was to explain how astronauts wrote about space exploration. The MLL approach revealed hidden strengths at the big picture (global or thematic) and close-up (local) levels of composing. Patterns of strengths discovered at the global and local levels are described with four case examples and instructional implications are noted. Suggestions for treatment are also provided. [ABSTRACT FROM AUTHOR]

This article explores the perceptual mismatches between Iranian English as a Foreign Language (EFL) learners and their teachers in the context of self-assessment of writing. The study found that providing learners with rating criteria can reduce perceptual mismatches and promote a more democratic classroom assessment. The text discusses the importance of self-assessment, the need for a common understanding of assessment criteria, and the benefits of involving students in the development and application of rubrics. The study also found that involving students in the assessment process and providing clear criteria can improve their understanding and accuracy in self-assessment. The document provides a list of references for further research on perceptions of feedback between students and teachers in English writing courses. [Extracted from the article]

While student feedback literacy has received increasing attention in second language education, little is known about the role of writing proficiency in the development of student feedback literacy. This qualitative study examined the characteristics of high and low English writing proficiency (i.e., HP, LP) students' feedback literacy in second language (L2) writing and identified the factors that shape these students' feedback literacy. This study involved five LP and five HP students with data collected from semi-structured interviews, students' drafts, teacher's written corrective feedback and peer feedback. The results found that both the LP and HP students were cognitively prepared in terms of appreciating feedback value with distinct self-orientation in feedback process and emotionally ready in regulating their affect for gains from handling feedback. Yet, the LP students were found less prepared to benefit from their current feedback practice due to their passive perspective of learner roles. By contrast, HP students displayed their preparedness featured by proactive learner roles in feedback process, well-developed judgement criteria and broader revisions. The internal consistency across dimensions of feedback literacy was also identified for HP students but not for LP students. Factors contributing to the disparity include students' perception of their writing proficiency, course engagement and their self-regulation ability. Pedagogical implications on enhancing feedback literacy for students with varied writing proficiency levels are provided. [ABSTRACT FROM AUTHOR]

This study explores lower secondary school teachers' arguments and perceptions for using a 26-point grading scale (26-PGS), and gender differences in assessment practice. An explanatory sequential design was conducted. First, teachers (n = 6) assessment of students' text (n = 182)was analysed.In the subsequent phase, an open-ended questionnaire with teachers (n = 54) was conducted and analysed. The study revealed that the teachers perceive that the 26-PGS provides precision. Teachers highlight the significance of using the 26-PGS as an alternative assessment method, aiming to foster students' growth, and as a message to motivate for learning. In addition, gender disparities in teachers' provision of grades and arguments of using a 26-PGS as part of their assessment practice were found. The study contributes to the existing literature by shedding light on teachers' assessment practice and gender differences regarding the use of grading scales and discuss its potential challenges in educational contexts. [ABSTRACT FROM AUTHOR]

Primary school teachers in England are responsible for the statutory assessment of writing, using a set of criteria. The aim of this research was to analyse the responses of teachers to the criteria at the expected standard. An elicitation exercise with 10 primary school teachers was conducted, where teachers talked through their interpretations of the criteria. Qualitative, inductive coding was undertaken to provide a rich and deep understanding of teachers' understandings of the assessment criteria. The perceptions of teachers are shown to be varied and complex. Definitions of the criteria were diverse, and fundamentally different concepts of writing were identified, which points to sources of misunderstanding. Limitations of the assessment criteria themselves are also highlighted, which caused disagreement among the teachers. The findings show evidence of threats to the validity and reliability of the assessment and its use as an accountability measure because of the lack of confidence in inferences drawn from the use of the criteria in the assessment of writing. [ABSTRACT FROM AUTHOR]

While much research supports the benefits of computer-mediated collaborative writing (CW) in second language (L2) classrooms, the assessment of CW has received scant attention. This study proposed an assessment scheme considering both the products and processes when assessing online synchronous CW, and explored its effects on learners' interactions and perceptions of the process-and-product-based assessment approach through qualitative analyses. Through analyzing 21 dyads' online pair talk and revision histories in CW tasks, post-task reflections, and interviews, this study found that assessment approach impacted patterns of collaboration, as well as the quantity and quality of peer interaction. Also, learners reported multiple advantages of the process-and-product-based assessment approach in computer-mediated CW, including increasing fairness, promoting better performance, and facilitating effective collaboration. Further, the process-and-product-based CW assessment approach raised learners' awareness of the value of the collaboration process, motivated students to be critical collaborators, and promoted regulated learning. Nevertheless, students expressed concerns that this assessment approach might bring pressure and disrupt idea negotiation. The findings can further our understanding of the role that assessment plays in computer-mediated CW tasks and shed new light on possible ways of implementing CW assessment practices that may facilitate learning and positive learner perceptions in L2 writing classrooms. 尽管许多研究表明计算机辅助合作写作在第二语言课堂中有很多益处, 但如何评价和评估合作写作却很少受到关注。本研究提出了一同时考虑写作结果和写作过程的对线上同步合作写作的评估方法, 并通过定性分析探讨了学习者对这一评估方法的态度以及在使用这一评估方法时对学习者合作互动的影响。通过对21组学习者合作写作时的在线合作对话, 修改历史记录, 任务后反思和访谈的分析, 本研究发现, 基于合作写作过程和结果的评估方法影响了学习者合作写作的互动模式以及同伴互动话语的数量和质量。此外, 学习者还报告了基于过程和结果的评估方法在计算机辅助合作写作中的多重优势, 包括提高公平性、促进更好的合作表现和提高写作质量。此外, 基于过程和结果的合作写作评估方法提高了学习者对合作写作过程价值的认知, 激励学生成为思辨的合作者, 并促进了学习者写作过程中自主调节学习。同时, 学习者也指出这种评估方法可能存在的问题, 例如, 可能给合作者带来压力或阻碍讨论的深度。总之, 该研究的结果可以进一步加深我们对写作测评在计算机辅合作写作任务中的作用的理解, 并为实施合作写作评估实践的可能方式提供新的视角, 这些实践可以促进二语写作学习和提高学习者的积极认知。. While much research discusses the benefits of writing collaboratively for second language learners in online settings, little research has studied how to grade learner performance when English learners complete collaborative writing (CW) tasks. We provided a method in this study to evaluate language learners' performance when they work in pairs to write essays online. This method considers both the quality of their co-constructed texts and the extent to which they actively participate in the discussions during the writing process. We also looked at how Chinese English learners perceived this grading method and how this method influenced their participation. Through analyzing 21 online pair talks and revision histories in CW tasks, post-task reflections, and interviews, we found that the grading method that we proposed impacted how much and how well students responded to each other and whether or not their active participation would solve the problems encountered in the CW process. Learners also reported many advantages of this grading method, including increasing fairness, promoting better writing performance, and facilitating effective collaboration. This grading method also helped the learners value the collaboration process, motivated them to be critical collaborators, and encouraged them to regulate their behaviors for better revisions in the writing process. Nevertheless, students expressed concerns that this grading method might bring pressure and disrupt their thinking while developing or negotiating ideas. [ABSTRACT FROM AUTHOR]

The contemporary practice of rating writing performances is grounded in an approach known as judging, which is done to avoid paying conscious attention to discrete elements in texts. Instead, it involves accounting for the overall impression made by a writing performance. However, studies have indicated that while this may be true on a conscious level, the concrete stimuli in texts still preconsciously influence the forming of such overall impressions. What is left largely unnoticed is that most assessment-relevant stimuli require the use of judging to be perceived as such. This implies that an overall, macro-judgment of a writing performance (expressed normally as a score) comprises individual (and largely preconsciously generated) micro-judgments coming together into a complex and non-linear combination-count. The paper presents an argument in favour of such a composition of judgments, demonstrates it empirically by means of a case study, and then discusses the wider consequences of this different perspective on judging.

The potential application of generative artificial intelligence (AI) in schools and universities poses great challenges, especially for the assessment of students’ texts. Previous research has shown that people generally have difficulty distinguishing AI-generated from human-written texts; however, the ability of teachers to identify an AI-generated text among student essays has not yet been investigated. Here we show in two experimental studies that novice (N = 89) and experienced teachers (N = 200) could not identify texts generated by ChatGPT among student-written texts. However, there are some indications that more experienced teachers made more differentiated and more accurate judgments. Furthermore, both groups were overconfident in their judgments. Effects of real and assumed source on quality assessment were heterogeneous. Our findings demonstrate that with relatively little prompting, current AI can generate texts that are not detectable for teachers, which poses a challenge to schools and universities in grading student essays. Our study provides empirical evidence for the current debate regarding exam strategies in schools and universities in light of the latest technological developments.

Artificial intelligence technology is becoming increasingly essential to education. The outbreak of COVID-19 in recent years has led many schools to launch online education. Automated online assessments have become a hot topic of interest, and an increasing number of researchers are studying Automated Essay Scoring (AES). This work seeks to summarise the characteristics of current AES systems used in English writing assessment, identify their strengths and weaknesses, and finally, analyse the limits of recent studies and research trends. Search strings were used to retrieve papers on AES systems from 2018 to 2023 from four databases, 104 of which were chosen to be potential to address the posed research aims after study selection and quality evaluation. It is concluded that the existing AES systems, although achieving good results in terms of accuracy in specific contexts, are unable to meet the needs of teachers and students in real teaching scenarios. The improvements of these systems relate to the scalability of the system for assessing different topics or styles of the essays, the accuracy of the model’s predicted scores, as well as the reliability of outcomes: improving the robustness of AES models with some adversarial inputs, the richness of AES system functionality, and the development of AES assist tools.

Despite the urgency of assessment instruments in teaching foreign language writing, evidence regarding how teachers develop assessment instruments and maintain their quality, particularly for comparison and contrast essay writing based on a genre-based approach to teachers’ classroom practices, is scantily reported. To address this empirical need, this article showcases how a set of teacher-made assessment instruments for a comparison and contrast essay based on a genre-based approach in a writing essay course at a public university in East Java was carefully and deliberately developed. To attain the research objective, this research enacted design-based research (DBR) inspired by Cavallaro and Sembiante (2021). In addition to content validity, face validity and inter-rater reliability were ensured by involving ten students, three fellow writing teachers, and two trained raters. This assessment instrument development project produces a set of valid formative assessment instruments, comprising a writing prompt and scoring rubric. The inter-rater reliability shows a high degree of positive correlation with a reliability coefficient of 0.862 from a correlational analysis based on Pearson correlation coefficient. This in-practice assessment instrument development can be used as a reference, contributing to the teachers’ assessment literacy development. It also serves as a valuable model for future researchers to develop and expand nested classroom assessment scholarship. A well-structured validity and reliability process of assessment instrument development will practically enhance the teaching practices in the essay writing course in particular and EFL teaching in general.

This study investigates the effect of interactional feedback on students’ writing skills. One hundred participants enrolled in an intermediate EFL course at the State University of Malang, Indonesia, were recruited for this research. The quantitative method was employed for data analysis. The primary data analysis method used was the ANCOVA test, followed by the Wilcoxon and Mann-Whitney tests. The results reveal that the dependent variables in the experimental group exhibited higher means compared to the control group. The ANCOVA test show that the dependent variables (writing length, accuracy, and effectiveness) were significantly affected by the addition of feedback (p = 0.000). However, no significant differences were found between the experimental and control groups regarding accuracy (p = 0.425) and writing length variables (p = 0.731). As a result, interactional feedback significantly impacted EFL students’ writing ability. This result highlights the need for thorough planning and preparation, including preparing ESL/EFL students through explicit instruction prior to peer review, to ensure that learners’ interactional feedback is useful. The findings suggest that EFL teachers should carefully select feedback styles that align with the intended purpose of providing feedback. For instance, more specific feedback options may prove more effective in assisting students in revising and improving their written assignments. Finally, this study provides valuable recommendations for further research in this field.

Purpose: We aimed to perform the first review of research focusing on written discourse performance in people with acquired neurogenic communication disorders. In studies from 2000 onward, we specifically sought to determine: (1) the differences between patient populations and control groups, (2) the differences between different patient populations, (3) longitudinal differences between patient populations, and (4) modality differences between spoken and written discourse performance. Methods: We completed a thorough search on MEDLINE, Embase, Cochrane, APAPsycinfo, Web of Science, and Scopus databases. We identified studies that focus on written discourse performance in people with aphasia, primary progressive aphasia, mild cognitive impairment, and Alzheimer's disease. Results: Nineteen studies were identified from the review of literature, some of which addressed more than one of our review questions. Fifteen studies included a comparison between clinical populations and controls. Six studies compared different characteristics of patient populations. Three studies reported changes over time in progressive disorders. Six studies targeted different modalities of discourse. Conclusion: Differences in linguistic features by patient populations are not yet clear due to the limited number of studies and different measures and tasks used across the studies. Nevertheless, there is substantial evidence of numerous linguistic features in acquired neurogenic communication disorders that depart from those of healthy controls. Compared to the controls, people with aphasia tend to produce fewer words, and syntactically simpler utterances compared to the controls. People with Alzheimer's disease produce less information content, and this feature increases over time, as reported in longitudinal studies. Our review imparts additional information that written and spoken discourse provide unique insights into the cognitive and linguistic deficits experienced by people with aphasia, Alzheimer's disease, mild cognitive impairment and primary progressive aphasia and provide targets for treatment to improve written communication in these groups. [ABSTRACT FROM AUTHOR]

In the past years, comparative assessment approaches have gained ground as a viable method to assess text quality. Instead of providing absolute scores to a text as in holistic or analytic scoring methods, raters in comparative assessments rate text quality by comparing texts either to pre-selected benchmarks representing different levels of writing quality (i.e., benchmark rating method) or by a series of pairwise comparisons to other texts in the sample (i.e., comparative judgement; CJ). In the present study, text quality scores from the benchmarking method and CJ are compared in terms of their reliability, convergent validity and scoring distribution. Results show that benchmark ratings and CJ-ratings were highly consistent and converged to the same construct of text quality. However, the distribution of benchmark ratings showed a central tendency. It is discussed how both methods can be integrated and used such that writing can be assessed reliably, validly, but also efficiently in both writing research and practice. [ABSTRACT FROM AUTHOR]

This paper presents an analysis of an authentic writing assessment to externalise the knowledge, skills, and attitudes necessary for developing such assessments. Specifically, the paper examines a technical proposal writing assignment developed and used as a continual assessment in an undergraduate engineering course by employing an authentic assessment framework as the analytical lens. The findings showed that the framework can serve as a valuable guide in developing authentic writing assessments. Further, it became evident that achieving functional authenticity in all dimensions may not always be viable. Developing fully functionally authentic writing assessments/assignments for beginners is not always advisable, as completing such assignments may lead to cognitive overload. Therefore, it is recommended that both the dimensions of authenticity and students' ability levels be considered when analysing, developing, and using authentic writing assessments and assignments. Indeed, striking the right balance between authenticity and students' cognitive capacity is crucial for creating optimal learning experiences. [ABSTRACT FROM AUTHOR]

This article discusses the alignment between Assessment for Learning (AfL) and Motivational Strategy (MotS) in the context of emergency remote learning and teaching English as a Foreign Language (EFL) writing. The study examines the practices of seven higher education teachers in Indonesia and finds that the greatest alignment occurs in the "maintain" stage of MotS, where teachers create a supportive classroom environment that embraces mistakes as a natural part of learning and involves students in self and peer assessment. However, the constrained context leads to divergent conceptions of what motivates students. The study suggests integrating AfL and motivational strategies to enhance engagement and improve writing achievement. The text also discusses the impact of emergency remote teaching on assessment practices in EFL classrooms during the COVID-19 pandemic, highlighting changes in assessment instruments, criteria, feedback delivery, and peer- and self-assessment. The need for further research on teachers' motivational strategies and the alignment of AfL and MotS during emergency remote teaching is emphasized. The study uses a mixed-method design, including interviews with EFL teachers, to investigate their AfL practices and alignment with MotS. The findings show that while all teachers engaged in some AfL practices, not all of them enacted all the principles of AfL. There was also some alignment between MotS and AfL, particularly in terms of creating a supportive classroom culture and involving students in assessment. However, the constrained context of online teaching during the pandemic affected the implementation of certain practices, such as [Extracted from the article]

Abstract It is widely believed that raters’ cognition is an important aspect of writing assessment, as it has both logical and temporal priority over scores. Based on a critical review of previous research in this area, it is found that raters’ cognition can be boiled to two fundamental issues: building text images and strategies for articulating scores. Compared to the scoring contexts of previous research, the TEM 8 integrated writing task scoring scale has unique features. It is urgent to know how raters build text images and how they articulate scores for text images in the specific context of rating TEM8 compositions. In order to answer these questions, the present study conducted qualitative research by considering raters as problem solvers in the light of problem-solving theory. Hence, 6 highly experienced raters were asked to verbalize their thoughts simultaneously while rating TEM 8 essays, supplemented by a retrospective interview. Analyzing the collected protocols, we found that with regard to research question 1, the raters went through two stages by setting building text images as isolated nodes and building holistic text images for each dimension as two sub-goals, respectively. In order to achieve the first sub-goal, raters used strategies such as single foci evaluating, diagnosing, and comparing; for the second sub-goal, they mainly used synthesizing and comparing. Regarding the second question, the results showed that they resorted to two groups of strategies: demarcating boundaries between scores within a dimension and discriminating between dimensions, each group consisting of more specific processes. Each of the extracted processes was defined clearly and their relationships were delineated, on the basis of which a new working model of the rating process was finalized. Overall, the present study deepens our understanding of rating processes and provides evidence for the scoring validity of the TEM 8 integrated writing test. It also provides implications for rating practice, such as the need for the distinction between two types of analytical rating scales.

Advances in writing technologies have fostered the gradual move from Paper-Based (PB) to Computer-Based (CB) writing assessment in foreign language (FL) education. The trend was further boosted by the outburst of the Covid-19 pandemic, which led to drastic changes in teaching and assessment practices. In higher education (HE) contexts, writing assessment in English as a foreign language (EFL) is more and more often managed by means of online learning platforms, where students create and submit written assignments on the computer. This paper examines a sample of 100 EFL exam essays completed by 100 Italian university students. Half of the essays were computer-typed on Moodle from home during the Covid-19 lockdown, whereas the other half were handwritten on paper during a face-to-face exam session after the Covid-19 emergency. The study presented herein compares the amount and types of formal errors in the CB and PB written assignments respectively. The results of the comparison may be useful to suggest hypotheses on the impact that both writing modality and environment can have on the quality of EFL writing in assessment contexts.

PurposeWe aimed to perform the first review of research focusing on written discourse performance in people with acquired neurogenic communication disorders. In studies from 2000 onward, we specifically sought to determine: (1) the differences between patient populations and control groups, (2) the differences between different patient populations, (3) longitudinal differences between patient populations, and (4) modality differences between spoken and written discourse performance.MethodsWe completed a thorough search on MEDLINE, Embase, Cochrane, APAPsycinfo, Web of Science, and Scopus databases. We identified studies that focus on written discourse performance in people with aphasia, primary progressive aphasia, mild cognitive impairment, and Alzheimer’s disease.ResultsNineteen studies were identified from the review of literature, some of which addressed more than one of our review questions. Fifteen studies included a comparison between clinical populations and controls. Six studies compared different characteristics of patient populations. Three studies reported changes over time in progressive disorders. Six studies targeted different modalities of discourse.ConclusionDifferences in linguistic features by patient populations are not yet clear due to the limited number of studies and different measures and tasks used across the studies. Nevertheless, there is substantial evidence of numerous linguistic features in acquired neurogenic communication disorders that depart from those of healthy controls. Compared to the controls, people with aphasia tend to produce fewer words, and syntactically simpler utterances compared to the controls. People with Alzheimer’s disease produce less information content, and this feature increases over time, as reported in longitudinal studies. Our review imparts additional information that written and spoken discourse provide unique insights into the cognitive and linguistic deficits experienced by people with aphasia, Alzheimer’s disease, mild cognitive impairment and primary progressive aphasia and provide targets for treatment to improve written communication in these groups.

It is widely believed that raters' cognition is an important aspect of writing assessment, as it has both logical and temporal priority over scores. Based on a critical review of previous research in this area, it is found that raters' cognition can be boiled to two fundamental issues: building text images and strategies for articulating scores. Compared to the scoring contexts of previous research, the TEM 8 integrated writing task scoring scale has unique features. It is urgent to know how raters build text images and how they articulate scores for text images in the specific context of rating TEM8 compositions. In order to answer these questions, the present study conducted qualitative research by considering raters as problem solvers in the light of problem-solving theory. Hence, 6 highly experienced raters were asked to verbalize their thoughts simultaneously while rating TEM 8 essays, supplemented by a retrospective interview. Analyzing the collected protocols, we found that with regard to research question 1, the raters went through two stages by setting building text images as isolated nodes and building holistic text images for each dimension as two sub-goals, respectively. In order to achieve the first sub-goal, raters used strategies such as single foci evaluating, diagnosing, and comparing; for the second sub-goal, they mainly used synthesizing and comparing. Regarding the second question, the results showed that they resorted to two groups of strategies: demarcating boundaries between scores within a dimension and discriminating between dimensions, each group consisting of more specific processes. Each of the extracted processes was defined clearly and their relationships were delineated, on the basis of which a new working model of the rating process was finalized. Overall, the present study deepens our understanding of rating processes and provides evidence for the scoring validity of the TEM 8 integrated writing test. It also provides implications for rating practice, such as the need for the distinction between two types of analytical rating scales. [ABSTRACT FROM AUTHOR]

The article reviews four empirical studies conducted on the University of Guam English Placement Test. The focus of the review ranges from updating the cloze test component and examining its relationship with writing proficiency, to applying an argument-based approach to investigate validity from the perspectives of raters. The intent of the article is to provide summaries of past studies, highlight their findings, and acknowledge their limitations, with the hope of serving as a reference for future researchers and assessment users. [ABSTRACT FROM AUTHOR]

Grading has long been the source of negative emotions in Writing Studies for teachers and students alike; these negative emotions and experiences especially affect women-identifying professors and professors of color. This poetic research study presents the found poems of writing instructors in one predominantly White Midwest US university English department; poems came from restories from participants' interviews about how they developed as assessors of writing. The aim is to foreground lived experiences of college professors who teach undergraduate writing for a living as the field continues to explore ethical and humane ways of assessment for all involved. [ABSTRACT FROM AUTHOR]

Restrictions and challenges brought on by the Covid-19 pandemic challenged higher education institutions to innovate to keep reaching teaching and learning goals. In South Africa, existing social inequalities were exacerbated by the pandemic restrictions and many students faced severe challenges in terms of access and support to aid in their academic success. Even under 'normal' circumstances, developing students' academic writing skills and critical thinking ability can be challenging. It is this teaching and learning goal, along with the added contextual challenges brought on by the pandemic that prompted the intervention described in this paper. How can we teach and assess critical thinking, and higher-order aspects of academic writing, in such a resource scarce environment? In this paper, the development and results of formative, multiple choice question (MCQ)-style, online tests aimed at developing critical thinking and higher-order aspects of academic writing is discussed. The development of the test through intentional test item design and online test setup is explained whereafter statistical analysis of student participation and results are presented. These preliminary results show that students generally use learning opportunities to their advantage to learn and improve their marks. It also shows that flexible test structure and a supportive test environment can promote equality of outcomes for students, regardless of the contextual challenges they face. [ABSTRACT FROM AUTHOR]

This study evaluated the construct validity of six scoring traits of an automated writing evaluation (AWE) system called MI Write. Persuasive essays (N = 100) written by students in grades 7 and 8 were randomized at the sentence-level using a script written with Python's NLTK module. Each persuasive essay was randomized 30 times (n = 3000 total randomizations), and the mean trait scores for each set of randomized iterations were compared to those of the control text across all traits. We were specifically interested in evaluating the effects of randomization on the high-level traits of idea development and organization. Given the rubrics and qualitative feedback provided by MI Write, we hypothesized that these high-level traits ought to be sensitive to sentence-level randomization (i.e., scores should decrease). Overall, complete randomizations did not consistently significantly impact trait scoring for these high-level writing traits. In fact, more than a third of the essays saw significant increases in one or both high-level traits despite randomization, indicating a disconnect between MI Write's formative feedback and its underlying constructs. Findings have implications for consumers and developers of AWE. [ABSTRACT FROM AUTHOR]

Copyright of Zeitschrift für Erziehungswissenschaft is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Restrictions and challenges brought on by the Covid-19 pandemic challenged higher education institutions to innovate to keep reaching teaching and learning goals. In South Africa, existing social inequalities were exacerbated by the pandemic restrictions and many students faced severe challenges in terms of access and support to aid in their academic success. Even under ‘normal’ circumstances, developing students’ academic writing skills and critical thinking ability can be challenging. It is this teaching and learning goal, along with the added contextual challenges brought on by the pandemic that prompted the intervention described in this paper. How can we teach and assess critical thinking, and higher-order aspects of academic writing, in such a resource scarce environment? In this paper, the development and results of formative, multiple choice question (MCQ)-style, online tests aimed at developing critical thinking and higher-order aspects of academic writing is discussed. The development of the test through intentional test item design and online test setup is explained whereafter statistical analysis of student participation and results are presented. These preliminary results show that students generally use learning opportunities to their advantage to learn and improve their marks. It also shows that flexible test structure and a supportive test environment can promote equality of outcomes for students, egardless of the contextual challenges they face.

Assessing students' writing performance is essential to adequately monitor and promote individual writing development, but it is also a challenge. The present research investigates a benchmark rating procedure for assessing texts written by upper-elementary students. In two studies we examined whether a benchmark rating procedure (1) leads to reliable and generalisable scores that converge with holistic and analytic ratings, and (2) can be used for rating texts varying in topic and genre. Results support evidence that benchmark ratings are a valid indicator of text quality as they converge with holistic and analytic scores. They are also associated with less rater variance and less task-specific variance, leading to reliable and generalisable ratings. Moreover, a benchmark scale can be used for rating different tasks with the same reliability, at least when texts are written in the same genre. Taken together, a benchmark rating procedure ensures meaningful and useful information on students' writing. [ABSTRACT FROM AUTHOR]

Automated Writing Evaluation (AWE) is one of the machine techniques used for assessing learners' writing. Recently, this technique has been widely implemented for improving learners' editing strategies. Several studies have been conducted to compare self-editing with peer editing. However, only a few studies have compared automated peer and self-editing. To fill this research gap, the present study implements AWE software, WRITER, for peer and self-editing. For this purpose, a pre-post quasi-experimental research design with convenience sampling is done for automated and non-automated editing of cause-effect essay writing. Arab, EFL learners of English, 44 in number, have been assigned to four groups: two peer and self-editing control groups and two automated peer and self-editing experimental groups. There is a triangulation of the quasi-experimental design with qualitative data from retrospective notes and questionnaire responses of the participants during and after automated editing. The quantitative data have been analyzed using non-parametric tests. The qualitative data have undergone thematic and content analysis. The results reveal that the AWE software has positively affected both the peer and self-editing experimental groups. However, no significant difference is detected between them. The analysis of the qualitative data reflects participants' positive evaluation of both the software and the automated peer and self-editing experience. [ABSTRACT FROM AUTHOR]

Through a series of narratives, this article highlights the experiences of six faculty who have incorporated varying approaches to labor-based contract grading in their courses at Texas A&M University-San Antonio, a Hispanic-Serving Institution. Each narrative offers a personal illustration of the faculty members' struggles, achievements, and realizations as they have adopted and adapted labor-based contract grading. Keywords: labor-based assessment, contract grading, narrative, Hispanic-Serving Institution, writing assessment

As youth increasingly access the public sphere and contribute to civic life through digital tools, scholars and educators are rethinking how civically engaged writing is taught, nurtured, and assessed. This article presents the conceptual underpinnings of the National Writing Project's Civically Engaged Writing Analysis Continuum (CEWAC), a new tool for assessing youth's civically engaged writing. It defines four attributes of civically engaged writing using qualitative analysis of expert interviews and literature: employs a public voice, advocates civic engagement or action, argues a position based on reasoning and evidence, and employs a structure to support a position. The article also presents reliability and validity evidence for CEWAC. The study finds that CEWAC has a moderate to high level of exact agreement and a high level of exact or adjacent agreement. Covariation analyses showed that, even with similar scoring patterns, CEWAC's attributes hold at least a moderate level of independence. This evidence, coupled with robust qualitative evidence around reliability and validity, establish CEWAC's strong technical properties. The findings suggest that CEWAC can be used both in research and in the classroom to make visible attributes of civically engaged writing often overlooked in traditional assessment frameworks. Keywords: public writing, civic engagement, writing assessment, rubric, reliability

Placement of multilingual writers within writing programs is an important and challenging issue. If students perceive that the placement process is rigid and unfair, this perception may affect their attitudes and motivation levels while taking courses in the writing program. The purpose of this study was to see whether a specific subgroup of students (n = 65) in a large university writing program for multilingual students could be successful if allowed to collaborate, with guidance, in their own placement. Various data were collected about these students in their first quarter after matriculating in the writing program: instructors' initial ratings, students' outcomes in their initial course (final portfolio scores and course grades), and students' satisfaction levels with their placement after they had completed the course (via a brief survey). These data were compared to another group of students (n = 65) who received similar placement scores but were not given the choice to move up or down a level. Findings indicated the pilot group was able to succeed at their chosen course level at levels comparable to the comparison group, and they were happy with their placement choices. Implications for placement processes in multilingual writing programs are discussed. Keywords: multilingual writers, writing assessment, directed self-placement, placement processes, student agency

Grading writing, or judging language, can be difficult. Asao B. Inoue's Labor-Based Grading Contracts problematizes traditional assessment practices that assess writing "quality". Inoue explains how this type of practice operates to reproduce White supremacy because language standards are tied to historical White racial formations. He suggests an alternative assessment method (e.g., grading contracts) that is based on labor and compassion. If you find yourself dissatisfied with classroom grading practices or wanting to understand how writing assessment can be constructed to do social justice work, then Inoue's Labor-Based Grading Contracts is a great read. Keywords: grading contracts, race, writing assessment, labor