p-value Problems? An Examination of Evidential Value in Criminology.

Objectives: This study aims to assess the evidential value of the knowledgebase in criminology after accounting for the presence of potential Type I errors. Methods: The present study examines the distribution of 1248 p-values (that inform 84 statistically significant outcomes across 26 systematic reviews) in meta-analyses on the topic of crime and justice published by the Campbell Collaboration (CC) using p-curve analysis. Results: The distribution of all CC p-values have a significant cluster of p-values immediately below 0.05, which is indicative of p-hacking. Evidential value (right skewed p-curves) is detected in most meta-analytic topic areas but not motivational interviewing (substance use outcome), sex offender treatment (sexual/general recidivism), police legitimacy (procedural justice), street-level drug law enforcement (total crime), and treatment effectiveness in secure corrections (juvenile recidivism). Conclusions: More studies, especially carefully designed and implemented randomized experiments with sufficiently large sample sizes, are needed before we are able to affirm the presence of evidential value and replicability of studies in all CC topic areas with confidence. [ABSTRACT FROM AUTHOR]