Search

We appreciate the thoughtful reactions of our colleagues to the “no prescriptives, please” proposal of Robinson et al. (2013), as well as the opportunity to respond to them. For the most part, our colleagues seem to agree that a problem exists in terms of unwarranted recommendations for practice appearing too often in educational research journals. At the same time, we realize that because our proposed policy is rather Draconian—in that it seeks to eliminate all recommendations for practice in all primary research journals—every responder mentioned that it was too extreme. This was intentional on our part. If, as Harris (2013) suggested, increased awareness and scrutiny become topics for further discussion among APA editors, then our modest mission will have been accomplished. In this rejoinder, we attempt to restate and clarify our proposal, along with its entailments, followed by brief comments on each responder's response.

Recommendations for practice are routinely included in articles that report educational research. Robinson et al. suggest that reports of primary research should not routinely do so. They argue that single primary research studies seldom have sufficient external validity to support claims about practice policy. In this article, I draw on recent statistical research that has formalized subjective notions about generalizability from experiments. I show that even rather large experiments often do not support generalizations to policy-relevant inference populations. This suggests that single primary studies are unlikely to be sufficiently generalizable to support recommendations for practice.

Robinson et al. (Educ Psychol Rev 25:291–302, 2013) have suggested refraining from practice and policy recommendations in primary educational research articles, in particular because primary research journals are not the appropriate outlet for such recommendations, the evidence provided by one research article is usually not sufficient, and making bold statements about practice in primary research trivializes the challenges of practice implementations. This commentary on Robinson et al. is a plea to continue providing such recommendations in primary research journals because much of educational research is not only theoretically but also practically grounded (i.e., use-inspired basic research). Contributions to the discourse on practice and theory are an intrinsic and productive part of educational research, even if some recommendations may turn out to be oversimplified or even wrong in further research. Nevertheless, there is agreement with Robinson et al. that we should usually not formulate general, sweeping recommendations. More modest and, therefore, legitimate types of practice recommendations are proposed, and conditions are discussed when such recommendations in primary research journals seem to be justified. Finally, some thoughts on how to communicate practice recommendations to practitioners were outlined.

Robinson et al. (Educ Psychol Rev 25(2):291-302, 2013) offer a thoughtful and powerful argument for a policy change for primary educational research journals. This policy change would “disallow recommendations for practice” (p. 10) in primary educational research journals. They provided compelling examples of works in which authors have either offered implications that go beyond their research design (i.e., stating causal relationships based on correlational data) or speculations and implications that go (way) beyond the results of their study. In my response, I first identify the many points made by Robinson et al. with which I agree and make it clear that I believe this discussion is a healthy one for the field. I do not, however, concur with the recommendation for an editorial policy change for primary research journals and offer reasons why the policy recommended is both too much and too little to deal with the issues they have identified as well as additional issues I find concerning in the submission and publication of studies in primary research journals.

In response to the call by Robinson et al. (25(2): 24–28, 2013) for a moratorium on recommendations for practice and policy in articles published in primary research journals, Alexander forwards four counterarguments that allow for what are termed reasoned and reasonable speculations. Among those counterarguments are the claim that (a) seeking influence in educational practice is a raison d'etre for educational psychology researchers; (b) problems evident in authors’ conclusions and implications are indicative of a deeper and broader issue; (c) speculations are unavailable and essential aspects of educational research; and (d) potential recommendations should be situated within the primary research document and with the data that justify their articulation.

To counteract what we see as a growing research-reporting concern, we propose the following editorial-policy change regarding the content of primary research articles in educational research journals: Contributors should restrict their discussion and conclusions to their data and not offer recommendations for educational practice nor speculate about the educational policy implications of their research. We believe that this change will be beneficial for maintaining a separation between evidence and opinion concerning the legitimate warrants of empirical research. We suggest viable accompaniments and alternatives to authors offering practical recommendations and policy speculations in primary-research journal articles. Among these is to rely on all sources of evidence, including systematic research syntheses based on the aggregated contribution of multiple research studies from different sources, with recommendations for practice tailored to specific situations so that both their validity and utility will be increased. It is hoped that our proposed policy change will initiate a conversation among researchers, editors, reviewers, and educational-policy experts with respect to the scope and purpose of primary research journal articles.

Questions have recently been raised about the value of statistical hypothesis testing, as well as the associated policy implications for publishing empirically based research in professional journals. In this Reflections note, we extend our (Robinson and Levin, 1997) earlier thoughts on what could, should, and should not be done to existing editorial practices.