When Is the Story in the Subgroups? Strategies for Interpreting and Reporting Intervention Effects on Subgroups. MDRC Working Papers on Research Methodology

This paper examines strategies for interpreting and reporting estimates of intervention effects for subgroups of a study sample. Specifically, the paper considers: why and how subgroup findings are important for applied research, the importance of pre-specifying sub- groups before analyses are conducted, the importance of using existing theory and prior research to distinguish between subgroups for whom study findings are confirmatory (hypothesis testing), as opposed to exploratory (hypothesis generating), and the conditions under which study findings should be considered confirmatory based on their pre-specification and pattern of statistical significance for the full sample, its subgroups, and their differences. These issues are illustrated by empirical examples from past work by the authors. (Contains 5 footnotes and 5 tables.) [This paper was also supported by funding from Judith Hispanic Fund for Methodological Innovation in Social Policy Research at MDRC.]