Impact of Verbal Scale Labels on the Elevation and Spread of Performance Ratings

In recent years many countries have introduced authentic performance-based assessments in their national exam systems. Teachers' ratings of their own candidates' performances may suffer from errors of leniency and range restriction. The goal of this study was to examine the impact of manipulating the descriptiveness, balancedness, and polarity of the rating scales on the elevation and spread of the performance ratings. The study was conducted in the field setting of a (simulated) high-stakes national exam in Dutch pre-vocational education. A total of 55 teachers rated the performances of 652 candidates (aged [plus or minus]16) on four authentic performance-based tasks. Multivariate multilevel analyses found the psychometric quality of the teachers' performance ratings to be more favorable for positively unbalanced scales than for balanced scales. Positively unbalanced rating scales yielded the lowest (i.e., least generous) and most discriminative ratings. The descriptiveness and polarity of the rating scales were of lesser importance for the rating distributions. On the basis of the findings it was decided to introduce positively unbalanced scales in the national exams for pre-vocational education. (Contains 2 tables.)