Progressing, not regressing: a possible solution to the problem of regression to the mean in unconscious processing studies

How convincing is current evidence for unconscious processing? Recently, a major criticism suggested that this evidence might be fully explained by a mere statistical phenomenon: regression to the mean (RttM). Since excluding participants based on an awareness assessment is a common practice in such studies, this post-hoc data selection might evoke RttM and lead to false effects that are driven by aware participants wrongfully classified as unaware. Here, we examined this criticism using both simulations and data from 15 studies probing unconscious processing (43 effects overall). In line with the original criticism, we confirmed that the reliability of awareness measures in the field is concerningly low. Yet using simulations we showed that reliability measures might be unsuitable for estimating error in awareness measures. Furthermore, we examined three proposed ways to assess whether an effect is genuine or reflects RttM; all suffered from substantial limitations, such as a lack of power or an unjustified linearity assumption. Accordingly, we suggest a new nonparametric solution, which enjoys high sensitivity and relatively high power. Together, this work emphasizes the need to account for the contribution of measurement error to effects of unconscious processing. It further suggests a way to meet the important challenge posed by RttM, in an attempt to establish a reliable and robust corpus of knowledge in studying unconscious processing.