An Empirical Investigation of Direct and Indirect Measures of Writing. Report of the 1980-81 Georgia Competency Based Education Writing Assessment Project.

Twenty-four high schools in 18 Georgia school systems were selected to participate in a project to check the potential validity, reliability, usefulness, and costs of several direct and indirect measures of writing: holistic, analytic, primary trait, writing mechanics (error counts), and indirect measures (objective multiple choice items). Another goal was to assess certain variables with different numbers of students and in different situations. Variables included types of tasks, rater training and background, scoring procedures and time, rater reliability, types of feedback, and developmental scoring costs per pupil tested. Each participating school received two tests--one writing sample that could be scored holistically, analytically, or by mechanics count, and either an objective test or a second writing sample to be scored for primary traits. The findings indicated that (1) holistic ratings were the main or most widely representative score, (2) objective tests could be scored cheaply, (3) holistic scoring was the most economical, and (4) analytic scoring was the most expensive. Based on the findings it was recommended that holistically rated samples be used as a measure of minimum competence and that a backup system be used in which some of the holistically rated papers are also rated analytically or with a carefully chosen objective measure. (HOD)