Back to Search Start Over

CAF indices and human ratings of oral performances in an opinion-based monologue task

Authors :
Chie Ogawa
Source :
Language Testing in Asia, Vol 12, Iss 1, Pp 1-18 (2022)
Publication Year :
2022
Publisher :
SpringerOpen, 2022.

Abstract

Abstract This study explored two assessment approaches to oral performances: analytical complexity, accuracy, and fluency (CAF) indices and human raters’ evaluations. CAF indices are frequently used in second-language speaking (L2) research; however, because tasks are communicative and goal-oriented, the degree to which students achieve such communicative goals must also be included. By incorporating human ratings of monologue organization and perceived CAF into speaking assessments, researchers can better understand the relationship between the analytical CAF indices and human ratings of a monologue task. The participants consisted of 48 English as a Foreign Language (EFL) students in a Japanese university. Their oral performances of 2-min opinion-based monologues were audio-recorded and then transcribed and analyzed using CAF measures. In addition, 11 human raters evaluated the same recordings in terms of the following criteria: topic organization, complexity, accuracy, and fluency. These ratings were then analyzed using the many-facet Rasch measurement (MFRM). Multiple linear regression results showed that fluency accounted for a significant amount of the human ratings, but other measures (lexis, complexity, accuracy) explained only a small portion of the variance. This study concluded with implications regarding L2 testing in speaking assessments.

Details

Language :
English
ISSN :
22290443
Volume :
12
Issue :
1
Database :
Directory of Open Access Journals
Journal :
Language Testing in Asia
Publication Type :
Academic Journal
Accession number :
edsdoj.193d823de0ba41e5b5a0f20c6d8e9f52
Document Type :
article
Full Text :
https://doi.org/10.1186/s40468-022-00154-9