Toward a Technology for Evaluating Evaluation.

The aim of this paper is to present a logical structure for the evaluation of evaluation (meta-evaluation) and to suggest ways of conducting such evaluations. Part I contains an analysis of background factors and problems associated with meta-evaluation--that is, the evaluation of evaluation. This part discusses the need for meta-evaluation and summarizes some of the pertinent literature. Suggestions are made concerning what criteria should guide the development of a meta-evaluation methodology. The final and major portion of Part I is an enumeration of 6 classes of problems that jeopardize methodology. The second part of the paper is a conceptual response to the first part. Part II contains a definition of meta-evaluation and a set of premises to undergird a conceptualization of meta-evaluation. Most of part two is devoted to a logical structure for designing meta-evaluation studies. The third part of the paper is an application of the logical structure presented in Part II. Basically Part III contains 5 meta-evaluation designs. Four of the designs are for use in guiding evaluation work, and the fifth is used in judging completed evaluation work. Taken together the three parts of the paper are intended to provide a partial response to the needs for conceptual and practical developments of meta-evaluation. (Author/MLP)