1. Utilizing Primary Study Quality in Meta-Analyses: A Step-by-Step Tutorial [Preprint]
- Author
-
Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Luxembourg Centre for Educational Testing (LUCET) [research center], Research Council of Norway (project number 331640) and Doctoral School in Humanities and Social Sciences of the University of Luxembourg. [sponsor], Scherer, Ronny, Emslander, Valentin, Faculty of Language and Literature, Humanities, Arts and Education (FLSHASE) > Luxembourg Centre for Educational Testing (LUCET) [research center], Research Council of Norway (project number 331640) and Doctoral School in Humanities and Social Sciences of the University of Luxembourg. [sponsor], Scherer, Ronny, and Emslander, Valentin
- Abstract
Evaluating the quality of primary studies is a key step in meta-analytic reviews to reduce the risk of bias and establish the validity of the meta-analytic inferences. However, the extant body of research offers little guidance on how to represent and incorporate primary study quality (PSQ) in meta-analyses, and some common procedures, such as creating sum scores from a set of quality indicators, often lack the backing from measurement models. Addressing these issues, we present a tutorial that guides meta-analysts in their analytic decisions and approaches to represent and incorporate PSQ. Specifically, we describe, review, and illustrate approaches to (a) select or create quality indicators or scores a priori or as part of the meta-analytic model; (b) examine the possible moderator effects of PSQ; and (c) test the sensitivity of moderator effects to PSQ. We illustrate these approaches with three examples and present a step-by-step tutorial with analytic code for researchers’ guidance. Overall, we argue for representing PSQ model-based if multiple quality indicators are available, the testing of moderator effects of PSQ on the effect sizes and their heterogeneity, and performing moderator sensitivity analyses.
- Published
- 2023