Toward More Natural Field Testing of Classroom Materials.

Issues associated with the evaluation of 61 educational curriculum units, which was conducted by the American Institutes for Research (AIR) are described. The units were adapted to the AIR Career Education Curriculum framework by creating 133 instructional units within which students could develop career skills in an orderly sequence. The evaluation process involved initial revision, field testing, and final revision of the units. During the initial revision, the needs of teachers, school administrators, and publishers were seen to include reduction in the length of units, inclusion of alternative learning activities, reduction in cost, replacement of copyrighted passages, and improvement in the appearance of materials. Obtaining information without arousing resentment or reluctance was described as the most difficult aspect of a naturalistic field test. As a result, teacher orientation workshops were conducted to introduce the curriculum units. School district responses were described as passive resistant, personally committed, aggressive resistant or group committed. Methods for obtaining unambiguous answers to test questions without disrupting the natural setting are described, and sample module and activity evaluation instruments are appended. Results included 454 teacher field test packets and 11,700 student evaluation forms. Computer-assisted handscoring of results is explained, as well as the development of total data sets for each module. (Author/JAC)