Comprehensive Standards-Based Data Collection: Essential for Valid Assessment of Program Impact.

To get a valid picture of program impact via comprehensive data collection, combined with a focus on standards, an evaluation of a professional development program for elementary and secondary level teachers of science was carried out at the University of Hawaii at Manoa. This paper presents selected examples from the program evaluation, designed to be both exceptionally comprehensive and focused on standards, and it discusses what the evaluators learned from the overall experience. The Standards-based Teacher Education through Partnerships (STEP) program was designed to empower teachers to become leaders in the standards-based movement. The program evaluation standards of the Joint Committee on Standards for Educational Evaluation were used to guide the evaluation design. Evaluators took a multidimensional assessment approach that included in-class case studies, videotapes of "best lessons," self-reports, in-class observations, student and teacher artifacts, teacher awards and recognitions, portfolios, performance tests, teacher-institute data, and student achievement data. Taking such a broad look at the program ensures that the essence of program impact is identified. Using multiple indicators of the achievement of project objectives allowed the triangulation and inclusion of what otherwise might be "fringe" indicators. Insights from the STEP program are summarized. (Contains 14 figures and 15 references.) (SLD)