Conventional and Newer Statistical Methods in Meta-Analysis.

The assumptions and consequences of applying conventional and newer statistical methods to meta-analytic data sets are reviewed. The application of the two approaches to a meta-analytic data set described by L. V. Hedges (1984) illustrates the differences. Hedges analyzed six studies of the effects of open education on student cooperation. The conventional way to test the hypothesis that treatment fidelity significantly influenced results is through a t-test for independent results. Hedges' more modern approach was to use a chi-square analog of the analysis of variance (ANOVA), a method that, in contrast to conventional statistics, found strong support for the hypothesized effect. Conventional ANOVA and newer techniques were also applied to a data set in which all studies were of the same size, with each assumed to have experimental and control groups containing 25 students each. The cell means and variances for Hedges' meta-analytic data set were reconstructed to determine the source of the difference in results between conventional and newer tests. It is concluded that conventional ANOVA is appropriate for use with meta-analytic data sets because conventional ANOVA uses the correct error term for testing the significance of effects of group factors. Newer meta-analytic methods are not recommended because of their use of an inappropriate error term. (SLD)