Insight Into Evaluation Practice: A Content Analysis of Designs and Methods Used in Evaluation Studies Published in North American Evaluation-Focused Journals.

To describe the recent practice of evaluation, specifically method and design choices, the authors performed a content analysis on 117 evaluation studies published in eight North American evaluation-focused journals for a 3-year period (2004—2006). The authors chose this time span because it follows the scientifically based research (SBR) movement, which prioritizes the use of randomized controlled trials (RCTs) to study programs and policies. The purpose of this study was to determine the designs and data collection methods reportedly used in evaluation practice in light of federal guidelines enacted prior to 2004. Results show that, in contrast to the movement, nonexperimental designs dominate the field, that mixed-methods approaches just barely nudged out qualitative methods for most commonly used and that the majority of studies where statistical significance was reported indicated mixed significance. [ABSTRACT FROM PUBLISHER]