Rater Sensitivity to Qualities of Lexis in Writing

An examination of the scores given using analytic rating scales can provide insight into how well the scales are measuring what they are supposed to be measuring. Many analytic rating scales include separate scales for lexis and grammar, yet the distinction between lexis and grammar has not been widely researched or defined. It is therefore unclear whether separate lexis and grammar scales are appropriate. For this reason, the lexis scores for 140 timed essays on a Japanese university English proficiency test were analyzed to determine what portion of variance is accounted for by different lexical qualities. In this test, which was administered to incoming university students at the beginning of the academic year, it was found that lexical accuracy is predictive of lexis scores. The lexis scores, however, are predicted by the scores on the grammar scale much more than range, frequency, or even accuracy of lexis in the essays. The difficulties in separating lexis from grammar when rating writing are discussed. Finally, limitations of the study and suggestions for further research are given. (Contains 2 footnotes and 4 tables.)