Large-Scale Secondary Data Analysis--Part 2: For Practitioners

As frequent consumers and disseminators of research, school psychologists have an ethical obligation to critically evaluate the findings of studies (National Association of School Psychologists, 2010); however, this can feel burdensome when studies are behind paywalls and require hours to properly scrutinize. Particularly when studies utilizing secondary analysis boast of large sample sizes, fancy statistical terms, and a complicated method section, it is easy to be seduced by the complexity of the data and methods, or simply the stature of the journal. That said, school psychologists should all bear in mind the aphorism that "extraordinary claims require extraordinary evidence." Findings from large-scale secondary analyses are not, in and of themselves, extraordinary. Critical consumerism by all readers is especially important given that the peer review process, though currently the best process available, is not as reliable or discerning as one might hope to believe. The authors argue here that one does not need a PhD in statistics to be able to critically evaluate this type of scientific article. This article, the second in a series on secondary data analysis, provides practitioners a primer to help them critically evaluate this type of research in an effort to support defensible and useful interpretations of published scholarship. [For "Large-Scale Secondary Data Analysis--Part 1: For Researchers," see EJ1239445.]