A Compendium of Textbook Views on Planned versus Post Hoc Tests.

This paper focuses on two methods of performing multiple comparisons (post hoc or unplanned versus a priori or planned) for determining exactly where statistically significant results arise when there are more than "k equals two" groups. Textbook views and recommendations regarding the use of these two types of multiple comparisons are summarized. Analysis of variance (ANOVA) and related methods remain popular, although other methods are sometimes more useful and although the frequency of use of these methods in published research is declining. Researchers conducting an ANOVA frequently conduct unplanned or post hoc tests to determine exactly where statistically significant group differences actually arise. Various methodologists have referred to this practice as "fishing","milking", or "data snooping". It is widely known that planned multiple comparisons have greater statistical power than do unplanned multiple comparisons. Secondly, and more importantly, planned comparisons tend to make the researcher think in advance. Graduate study in statistics should reflect the importance of planned versus post hoc statistical design. In addition, there is a need for further investigation of current statistics textbooks being used in the different schools of education, business, and the behavioral sciences to determine if the issue of planned versus post hoc tests is being adequately addressed in instructional materials. A 63-item list of references, 4 data tables, and a list of selected computer control cards are included. (TJH)