551. Confidence Scoring of Speaking Performance: How Does Fuzziness become Exact?
- Author
-
Jin, Tan, Mak, Barley, and Zhou, Pei
- Abstract
The fuzziness of assessing second language speaking performance raises two difficulties in scoring speaking performance: "indistinction between adjacent levels" and "overlap between scales". To address these two problems, this article proposes a new approach, "confidence scoring", to deal with such fuzziness, leading to "confidence" scores between two adjacent levels applied to three scales. Since confidence scores have to be transformed to "an exact" score for test interpretation and use, membership functions and rule bases are applied and a confidence scoring algorithm is developed. Confidence scoring is demonstrated in the paper by an example to facilitate easy understanding. The paper then describes a pilot study that was conducted to try out the confidence scoring design. Initial results reveal that: first, confidence scoring is as feasible as traditional scoring; second, confidence scoring performs better in scoring dependability and in correlations with established benchmarks. At the end of the article, further studies are called for in order to build a validity argument and make further revisions to the confidence scoring method described here. (Contains 8 tables and 9 figures.)
- Published
- 2012
- Full Text
- View/download PDF