Search

The purpose of this study is to investigate whether providing preprint readers with information about preprint authors’ conflicts of interest (COIs) alters the perceived trust/credibility of preprints (as measured by download behavior), and whether this depends on the state of the COI (report no conflicts vs. report conflicts)

In this task, researchers will code some characteristics of registered reports and non-registered reports (focusing only on the last study reported in each article).

This component contains the data and analysis scripts for the overarching examination of Many Labs 5.

Supplemental material, PSPB_916631_Corrected_SUPPLEMENT for Meta-Analytic Use of Balanced Identity Theory to Validate the Implicit Association Test by Dario Cvencek, Andrew N. Meltzoff, Craig D. Maddox, Brian A. Nosek, Laurie A. Rudman, Thierry Devos, Yarrow Dunham, Andrew S. Baron, Melanie C. Steffens, Kristin Lane, Javier Horcajo, Leslie Ashburn-Nardo, Amanda Quinby, Sameer B. Srivastava, Kathleen Schmidt, Eugene Aidman, Emilie Tang, Shelly Farnham, Deborah S. Mellott, Mahzarin R. Banaji and Anthony G. Greenwald in Personality and Social Psychology Bulletin

Supplemental material, ML5_Overview_Open_Practices_Disclosure_Rev for Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability by Charles R. Ebersole, Maya B. Mathur, Erica Baranski, Diane-Jo Bart-Plange, Nicholas R. Buttrick, Christopher R. Chartier, Katherine S. Corker, Martin Corley, Joshua K. Hartshorne, Hans IJzerman, Ljiljana B. Lazarević, Hugh Rabagliati, Ivan Ropovik, Balazs Aczel, Lena F. Aeschbach, Luca Andrighetto, Jack D. Arnal, Holly Arrow, Peter Babincak, Bence E. Bakos, Gabriel Baník, Ernest Baskin, Radomir Belopavlović, Michael H. Bernstein, Michał Białek, Nicholas G. Bloxsom, Bojana Bodroža, Diane B. V. Bonfiglio, Leanne Boucher, Florian Brühlmann, Claudia C. Brumbaugh, Erica Casini, Yiling Chen, Carlo Chiorri, William J. Chopik, Oliver Christ, Antonia M. Ciunci, Heather M. Claypool, Sean Coary, Marija V. Čolić, W. Matthew Collins, Paul G. Curran, Chris R. Day, Benjamin Dering, Anna Dreber, John E. Edlund, Filipe Falcão, Anna Fedor, Lily Feinberg, Ian R. Ferguson, Máire Ford, Michael C. Frank, Emily Fryberger, Alexander Garinther, Katarzyna Gawryluk, Kayla Ashbaugh, Mauro Giacomantonio, Steffen R. Giessner, Jon E. Grahe, Rosanna E. Guadagno, Ewa Hałasa, Peter J. B. Hancock, Rias A. Hilliard, Joachim Hüffmeier, Sean Hughes, Katarzyna Idzikowska, Michael Inzlicht, Alan Jern, William Jiménez-Leal, Magnus Johannesson, Jennifer A. Joy-Gaba, Mathias Kauff, Danielle J. Kellier, Grecia Kessinger, Mallory C. Kidwell, Amanda M. Kimbrough, Josiah P. J. King, Vanessa S. Kolb, Sabina Kołodziej, Marton Kovacs, Karolina Krasuska, Sue Kraus, Lacy E. Krueger, Katarzyna Kuchno, Caio Ambrosio Lage, Eleanor V. Langford, Carmel A. Levitan, Tiago Jessé Souza de Lima, Hause Lin, Samuel Lins, Jia E. Loy, Dylan Manfredi, Łukasz Markiewicz, Madhavi Menon, Brett Mercier, Mitchell Metzger, Venus Meyet, Ailsa E. Millen, Jeremy K. Miller, Andres Montealegre, Don A. Moore, Rafał Muda, Gideon Nave, Austin Lee Nichols, Sarah A. Novak, Christian Nunnally, Ana Orlić, Anna Palinkas, Angelo Panno, Kimberly P. Parks, Ivana Pedović, Emilian Pękala, Matthew R. Penner, Sebastiaan Pessers, Boban Petrović, Thomas Pfeiffer, Damian Pieńkosz, Emanuele Preti, Danka Purić, Tiago Ramos, Jonathan Ravid, Timothy S. Razza, Katrin Rentzsch, Juliette Richetin, Sean C. Rife, Anna Dalla Rosa, Kaylis Hase Rudy, Janos Salamon, Blair Saunders, Przemysław Sawicki, Kathleen Schmidt, Kurt Schuepfer, Thomas Schultze, Stefan Schulz-Hardt, Astrid Schütz, Ani N. Shabazian, Rachel L. Shubella, Adam Siegel, Rúben Silva, Barbara Sioma, Lauren Skorb, Luana Elayne Cunha de Souza, Sara Steegen, L. A. R. Stein, R. Weylin Sternglanz, Darko Stojilović, Daniel Storage, Gavin Brent Sullivan, Barnabas Szaszi, Peter Szecsi, Orsolya Szöke, Attila Szuts, Manuela Thomae, Natasha D. Tidwell, Carly Tocco, Ann-Kathrin Torka, Francis Tuerlinckx, Wolf Vanpaemel, Leigh Ann Vaughn, Michelangelo Vianello, Domenico Viganola, Maria Vlachou, Ryan J. Walker, Sophia C. Weissgerber, Aaron L. Wichman, Bradford J. Wiggins, Daniel Wolf, Michael J. Wood, David Zealley, Iris Žeželj, Mark Zrubka and Brian A. Nosek in Advances in Methods and Practices in Psychological Science

Supplemental material, ML5_Overview_Supplemental_Material for Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability by Charles R. Ebersole, Maya B. Mathur, Erica Baranski, Diane-Jo Bart-Plange, Nicholas R. Buttrick, Christopher R. Chartier, Katherine S. Corker, Martin Corley, Joshua K. Hartshorne, Hans IJzerman, Ljiljana B. Lazarević, Hugh Rabagliati, Ivan Ropovik, Balazs Aczel, Lena F. Aeschbach, Luca Andrighetto, Jack D. Arnal, Holly Arrow, Peter Babincak, Bence E. Bakos, Gabriel Baník, Ernest Baskin, Radomir Belopavlović, Michael H. Bernstein, Michał Białek, Nicholas G. Bloxsom, Bojana Bodroža, Diane B. V. Bonfiglio, Leanne Boucher, Florian Brühlmann, Claudia C. Brumbaugh, Erica Casini, Yiling Chen, Carlo Chiorri, William J. Chopik, Oliver Christ, Antonia M. Ciunci, Heather M. Claypool, Sean Coary, Marija V. Čolić, W. Matthew Collins, Paul G. Curran, Chris R. Day, Benjamin Dering, Anna Dreber, John E. Edlund, Filipe Falcão, Anna Fedor, Lily Feinberg, Ian R. Ferguson, Máire Ford, Michael C. Frank, Emily Fryberger, Alexander Garinther, Katarzyna Gawryluk, Kayla Ashbaugh, Mauro Giacomantonio, Steffen R. Giessner, Jon E. Grahe, Rosanna E. Guadagno, Ewa Hałasa, Peter J. B. Hancock, Rias A. Hilliard, Joachim Hüffmeier, Sean Hughes, Katarzyna Idzikowska, Michael Inzlicht, Alan Jern, William Jiménez-Leal, Magnus Johannesson, Jennifer A. Joy-Gaba, Mathias Kauff, Danielle J. Kellier, Grecia Kessinger, Mallory C. Kidwell, Amanda M. Kimbrough, Josiah P. J. King, Vanessa S. Kolb, Sabina Kołodziej, Marton Kovacs, Karolina Krasuska, Sue Kraus, Lacy E. Krueger, Katarzyna Kuchno, Caio Ambrosio Lage, Eleanor V. Langford, Carmel A. Levitan, Tiago Jessé Souza de Lima, Hause Lin, Samuel Lins, Jia E. Loy, Dylan Manfredi, Łukasz Markiewicz, Madhavi Menon, Brett Mercier, Mitchell Metzger, Venus Meyet, Ailsa E. Millen, Jeremy K. Miller, Andres Montealegre, Don A. Moore, Rafał Muda, Gideon Nave, Austin Lee Nichols, Sarah A. Novak, Christian Nunnally, Ana Orlić, Anna Palinkas, Angelo Panno, Kimberly P. Parks, Ivana Pedović, Emilian Pękala, Matthew R. Penner, Sebastiaan Pessers, Boban Petrović, Thomas Pfeiffer, Damian Pieńkosz, Emanuele Preti, Danka Purić, Tiago Ramos, Jonathan Ravid, Timothy S. Razza, Katrin Rentzsch, Juliette Richetin, Sean C. Rife, Anna Dalla Rosa, Kaylis Hase Rudy, Janos Salamon, Blair Saunders, Przemysław Sawicki, Kathleen Schmidt, Kurt Schuepfer, Thomas Schultze, Stefan Schulz-Hardt, Astrid Schütz, Ani N. Shabazian, Rachel L. Shubella, Adam Siegel, Rúben Silva, Barbara Sioma, Lauren Skorb, Luana Elayne Cunha de Souza, Sara Steegen, L. A. R. Stein, R. Weylin Sternglanz, Darko Stojilović, Daniel Storage, Gavin Brent Sullivan, Barnabas Szaszi, Peter Szecsi, Orsolya Szöke, Attila Szuts, Manuela Thomae, Natasha D. Tidwell, Carly Tocco, Ann-Kathrin Torka, Francis Tuerlinckx, Wolf Vanpaemel, Leigh Ann Vaughn, Michelangelo Vianello, Domenico Viganola, Maria Vlachou, Ryan J. Walker, Sophia C. Weissgerber, Aaron L. Wichman, Bradford J. Wiggins, Daniel Wolf, Michael J. Wood, David Zealley, Iris Žeželj, Mark Zrubka and Brian A. Nosek in Advances in Methods and Practices in Psychological Science

Supplemental material, Uhlmann_Supplemental_Materials for Scientific Utopia III: Crowdsourcing Science by Eric Luis Uhlmann, Charles R. Ebersole, Christopher R. Chartier, Timothy M. Errington, Mallory Kidwell, Calvin K. Lai, Randy J. McCarthy, Amy Riegelman, Raphael Silberzahn and Brian A. Nosek in Perspectives on Psychological Science

Supplemental Material, SPPS829059_suppl_mat for (Ideo)Logical Reasoning: Ideology Impairs Sound Reasoning by Anup Gampa, Sean P. Wojcik, Matt Motyl, Brian A. Nosek, and Peter H. Ditto in Social Psychological and Personality Science

Individual differences in general speed lead to a positive correlation between the mean and standard deviation of mean latency. This “coarse” scaling effect causes the mean latency difference (MLD) to be spuriously correlated with general speed. Within individuals, the correlation between the mean and standard deviation of trial latencies leads contrasted distributions to increase their overlap as an MLD of fixed width is translated to the right. To address this “fine” scaling effect, contrasts based on within subject latency transformations including the logarithm, standardization, and ranking were evaluated and turned out to be distinctly superior to the MLD. Notably, the mean gaussian rank latency difference was internally consistent, eliminated fine scaling, meliorated coarse scaling, reduced correlations with general speed, increased statistical power to detect within subject and between group effects, and has the potential to increase the validity of inferences drawn from response latency data.

Social judgment is shaped by multiple biases operating simultaneously, but most bias-reduction interventions target only a single social category. In seven pre-registered studies (Total N > 7,000), we investigated whether asking participants to avoid one social bias impacted that and other social biases. Participants selected honor society applicants based on academic credentials. Applicants also differed on social categories irrelevant for selection: attractiveness and ingroup status. Participants asked to avoid potential bias in one social category showed small but reliable reductions in bias for that category (r = .095), but showed near zero bias reduction on the unmentioned social category (r = .006). Asking participants to avoid many possible social biases or alerting them to bias without specifically identifying a category did not consistently reduce bias. The effectiveness of interventions for reducing social biases may be highly specific, perhaps even contingent on explicitly and narrowly identifying the potential source of bias.

"We propose to change the default P-value threshold forstatistical significance for claims of new discoveries from 0.05 to 0.005."

We examined how felt age and desired age differed from chronological age across the age span. With each passing Earth year, felt and desired age do grow older, it just takes longer for the year to go by. Past age 25 or so, subjective aging appears to occur on Mars, where one Earth decade equals only 5.3 Martian years. In some sense, our minds age more slowly than our bodies do.

Supplemental material, Klein_Open_Practices_Disclosure for Many Labs 2: Investigating Variation in Replicability Across Samples and Settings by Richard A. Klein, Michelangelo Vianello, Fred Hasselman, Byron G. Adams, Reginald B. Adams, Sinan Alper, Mark Aveyard, Jordan R. Axt, Mayowa T. Babalola, Štěpán Bahník, Rishtee Batra, Mihály Berkics, Michael J. Bernstein, Daniel R. Berry, Olga Bialobrzeska, Evans Dami Binan, Konrad Bocian, Mark J. Brandt, Robert Busching, Anna Cabak Rédei, Huajian Cai, Fanny Cambier, Katarzyna Cantarero, Cheryl L. Carmichael, Francisco Ceric, Jesse Chandler, Jen-Ho Chang, Armand Chatard, Eva E. Chen, Winnee Cheong, David C. Cicero, Sharon Coen, Jennifer A. Coleman, Brian Collisson, Morgan A. Conway, Katherine S. Corker, Paul G. Curran, Fiery Cushman, Zubairu K. Dagona, Ilker Dalgar, Anna Dalla Rosa, William E. Davis, Maaike de Bruijn, Leander De Schutter, Thierry Devos, Marieke de Vries, Canay Doğulu, Nerisa Dozo, Kristin Nicole Dukes, Yarrow Dunham, Kevin Durrheim, Charles R. Ebersole, John E. Edlund, Anja Eller, Alexander Scott English, Carolyn Finck, Natalia Frankowska, Miguel-Ángel Freyre, Mike Friedman, Elisa Maria Galliani, Joshua C. Gandi, Tanuka Ghoshal, Steffen R. Giessner, Tripat Gill, Timo Gnambs, Ángel Gómez, Roberto González, Jesse Graham, Jon E. Grahe, Ivan Grahek, Eva G. T. Green, Kakul Hai, Matthew Haigh, Elizabeth L. Haines, Michael P. Hall, Marie E. Heffernan, Joshua A. Hicks, Petr Houdek, Jeffrey R. Huntsinger, Ho Phi Huynh, Hans IJzerman, Yoel Inbar, Åse H. Innes-Ker, William Jiménez-Leal, Melissa-Sue John, Jennifer A. Joy-Gaba, Roza G. Kamiloğlu, Heather Barry Kappes, Serdar Karabati, Haruna Karick, Victor N. Keller, Anna Kende, Nicolas Kervyn, Goran Knežević, Carrie Kovacs, Lacy E. Krueger, German Kurapov, Jamie Kurtz, Daniël Lakens, Ljiljana B. Lazarević, Carmel A. Levitan, Neil A. Lewis, Samuel Lins, Nikolette P. Lipsey, Joy E. Losee, Esther Maassen, Angela T. Maitner, Winfrida Malingumu, Robyn K. Mallett, Satia A. Marotta, Janko Međedović, Fernando Mena-Pacheco, Taciano L. Milfont, Wendy L. Morris, Sean C. Murphy, Andriy Myachykov, Nick Neave, Koen Neijenhuijs, Anthony J. Nelson, Félix Neto, Austin Lee Nichols, Aaron Ocampo, Susan L. O’Donnell, Haruka Oikawa, Masanori Oikawa, Elsie Ong, Gábor Orosz, Malgorzata Osowiecka, Grant Packard, Rolando Pérez-Sánchez, Boban Petrović, Ronaldo Pilati, Brad Pinter, Lysandra Podesta, Gabrielle Pogge, Monique M. H. Pollmann, Abraham M. Rutchick, Patricio Saavedra, Alexander K. Saeri, Erika Salomon, Kathleen Schmidt, Felix D. Schönbrodt, Maciej B. Sekerdej, David Sirlopú, Jeanine L. M. Skorinko, Michael A. Smith, Vanessa Smith-Castro, Karin C. H. J. Smolders, Agata Sobkow, Walter Sowden, Philipp Spachtholz, Manini Srivastava, Troy G. Steiner, Jeroen Stouten, Chris N. H. Street, Oskar K. Sundfelt, Stephanie Szeto, Ewa Szumowska, Andrew C. W. Tang, Norbert Tanzer, Morgan J. Tear, Jordan Theriault, Manuela Thomae, David Torres, Jakub Traczyk, Joshua M. Tybur, Adrienn Ujhelyi, Robbie C. M. van Aert, Marcel A. L. M. van Assen, Marije van der Hulst, Paul A. M. van Lange, Anna Elisabeth van ’t Veer, Alejandro Vásquez- Echeverría, Leigh Ann Vaughn, Alexandra Vázquez, Luis Diego Vega, Catherine Verniers, Mark Verschoor, Ingrid P. J. Voermans, Marek A. Vranka, Cheryl Welch, Aaron L. Wichman, Lisa A. Williams, Michael Wood, Julie A. Woodzicka, Marta K. Wronska, Liane Young, John M. Zelenski, Zeng Zhijia and Brian A. Nosek in Advances in Methods and Practices in Psychological Science

Twenty-nine teams involving 61 analysts used the same data set to address the same research question: whether soccer referees are more likely to give red cards to dark-skin-toned players than to light-skin-toned players. Analytic approaches varied widely across the teams, and the estimated effect sizes ranged from 0.89 to 2.93 ( Mdn = 1.31) in odds-ratio units. Twenty teams (69%) found a statistically significant positive effect, and 9 teams (31%) did not observe a significant relationship. Overall, the 29 different analyses used 21 unique combinations of covariates. Neither analysts’ prior beliefs about the effect of interest nor their level of expertise readily explained the variation in the outcomes of the analyses. Peer ratings of the quality of the analyses also did not account for the variability. These findings suggest that significant variation in the results of analyses of complex data may be difficult to avoid, even by experts with honest intentions. Crowdsourcing data analysis, a strategy in which numerous research teams are recruited to simultaneously investigate the same research question, makes transparent how defensible, yet subjective, analytic choices influence research results.

Indirect evaluation measures are used as a dependent measure to assess the impact of experimental interventions on shifting pre-existing attitudes or creating new attitudes. In four experiments (total N = 13,894), we compared the sensitivity of four indirect evaluation measures to evaluative information about two novel targets. Evaluative sensitivity was strongest for the Implicit Association Test (IAT). Other measures were more similar in their sensitivity, but the pattern, from stronger to the weakest was the Affect Misattribution Procedure (AMP), the Sorting Paired Features (SPF), and then Evaluative Priming task (EPT). To the extent that these findings are generalizable to related research applications, these results suggest that the measures differ in their research efficiency (power). For example, to achieve 80% power to detect the evaluative learning effect in the present studies, direct self-report would require 10 participants, the IAT 28 participants, the AMP 57, the SPF 79, and the EPT 131.

This piece originally appeared on Retraction Watch: http://retractionwatch.com/2016/03/07/lets-not-mischaracterize-replication-studies-authors/. It is a response to D. Gilbert, King, Pettigrew, and Wilson's (2016) claims that low replication rates in the Reproducibility Project: Psychology were due to differences between the original and replication methodologies. Nosek and E. Gilbert use an example of D. Gilbert's characterizations of a study to suggest that the criticism is misleading at best.

In their review of validity of Implicit Association Test and affective priming, De Houwer, Teige-Mocigemba, Spruyt, and Moors (2009) identified validity with establishment of “basic theoretical understanding” of the measures. We agree that theoretical understanding has an important role in making measures more valid and useful. Nevertheless, we conclude that scientific advancement will more often be well served by prioritizing pragmatic goals of establishing the measures’ predictive validity and their adequate sensitivity to individual differences.

Existing norms for scientific communication are rooted in anachronistic practices of bygone eras, making them needlessly inefficient. We outline a path that moves away from the existing model of scientific communication to improve the efficiency in meeting the purpose of public science – knowledge accumulation. We call for six changes: (1) full embrace of digital communication, (2) open access to all published research, (3) disentangling publication from evaluation, (4) breaking the “one article, one journal” model with a grading system for evaluation and diversified dissemination outlets, (5) publishing peer review, and, (6) allowing open, continuous peer review. We address conceptual and practical barriers to change, and provide examples showing how the suggested practices are being used already. The critical barriers to change are not technical or financial; they are social. While scientists guard the status quo, they also have the power to change it.

Disgust is linked to social evaluation. People with higher disgust sensitivity exhibit more sexual prejudice, and inducing disgust increases sexual prejudice. We tested whether inducing moral elevation, the theoretical opposite of disgust, would reduce sexual prejudice. In four studies (N = 3622), we induced elevation with inspiring videos and then measured sexual prejudice with implicit and explicit measures. Compared to control videos that elicited no particular affective state, we found that elevation reduced implicit and explicit sexual prejudice, albeit very slightly. No effect was observed when the target of social evaluation was changed to race (Black-White). Inducing amusement, another positive emotion, did not significantly affect sexual prejudice. We conclude that elevation weakly but reliably reduces prejudice towards gay men.

14 PI Demo site IAT study data from 2002 to current

Open Science Collaboration (in press). Maximizing the reproducibility of your research. In S. O. Lilienfeld & I. D. Waldman (Eds.), Psychological Science Under Scrutiny: Recent Challenges and Proposed Solutions. New York, NY: Wiley.

In reporting Implicit Association Test (IAT) results, researchers have most often used scoring conventions described in the first publication of the IAT (A. G. Greenwald, D. E. McGhee, & J. L. K. Schwartz, 1998). Demonstration IATs available on the Internet have produced large data sets that were used here to evaluate alternative scoring procedures. Candidate new algorithms were examined in terms of their (a) correlations with parallel self- report measures, (b) resistance to an artifact associated with speed of responding, (c) internal consistency, (d) sensitivity to known influences on IAT measures, and (e) resistance to known procedural influences. The best-performing measure incorporates data from the IAT’s practice trials, uses a metric that is calibrated by each respondent’s latency variability, and includes a latency penalty for errors. This new algorithm strongly outperforms the earlier (conventional) procedure.

Implicit preferences are malleable, but does that change last? We tested 9 interventions (8 real and 1 sham) to reduce implicit racial preferences over time. In 2 studies with a total of 6,321 participants, all 9 interventions immediately reduced implicit preferences. However, none were effective after a delay of several hours to several days. We also found that these interventions did not change explicit racial preferences and were not reliably moderated by motivations to respond without prejudice. Short-term malleability in implicit preferences does not necessarily lead to long-term change, raising new questions about the flexibility and stability of implicit preferences.

Many areas of social psychological research investigate how social information may bias judgment. However, most measures of social judgment biases are [1] low in reliability because they use a single response, [2] not indicative of individual differences in bias because they use between-subjects designs, [3] inflexible because they are designed for a particular domain, and [4] ambiguous about magnitude of bias because there is no objectively correct answer. We developed a measure of social judgment bias, the Judgment Bias Task, in which participants judge profiles varying in quality for a certain outcome based on objective criteria. The presence of ostensibly irrelevant social information provides opportunity to assess the extent to which social biases undermine the use of objective criteria in judgment. The JBT facilitates measurement of social judgment biases by [1] using multiple responses, [2] indicating individual differences by using within-subject designs, [3] being adaptable for assessing a variety of judgments, [4] identifying an objective magnitude of bias, and [5] taking six minutes to complete on average. In nine pre-registered studies (N> 9,000) we use the JBT to reveal two prominent social judgment biases: favoritism towards more physically attractive people and towards members of one’s ingroup. We observe that the JBT can reveal social biases, and that these sometimes occur even when the participant did not intend or believe they showed biased judgment. A flexible, objective, efficient assessment of social judgment biases will accelerate theoretical and empirical progress.

We compared the psychometric qualities of seven indirect attitude measures across three attitude domains (race, politics and self-esteem) with a large sample (n = 23,413). We compared the measures on internal consistency, sensitivity to known effects, relationship with indirect and direct measures of the same topic, reliability and validity of single-category attitude measurement, ability to detect meaningful variance among people with non-extreme attitudes, and robustness to the exclusion of misbehaved or well-behaved participants. All seven indirect measures correlated with each other, and with direct measures of the same topic. These relations were always weak for self-esteem, moderate for race and strong for politics. This pattern suggests that some of the source of variation in reliability and predictive validity of indirect measures is a function of the concepts rather than the methods. The Implicit Association Test (IAT) and Brief IAT (BIAT) showed the best overall psychometric quality, followed by the Go-No go Association Task, Single-target IAT (ST-IAT), Affective Misattribution Procedure (AMP), Sorting Paired Features task, and Evaluative Priming. The AMP showed a steep decline in its psychometric qualities when people with extreme attitude scores were removed. Single-category attitude scores computed for the IAT and BIAT showed good relationship with other attitude measures, but no evidence of discriminant validity between paired categories. The other measures, especially the AMP and ST-IAT, showed better evidence for discriminant validity. The results inform on the validity of measures as attitude assessments, but do not speak to the implicitness of the measured constructs.

Respondents at an Internet website completed over 600,000 Implicit Association Tests (IATs) between October 1998 and April 2000 to measure attitudes toward and stereotypes of social groups (www.yale.edu/implicit). Their responses demonstrated, on average, implicit preference for white over black and young over old, and stereotypic associations linking male with science and career, and female with liberal arts and family. The main purpose was to provide a demonstration site at which respondents could experience their implicit attitudes and stereotypes toward social groups. Nevertheless, the data collected are rich in information regarding the operation of attitudes and stereotypes, most notably the strength of implicit attitudes, the association and dissociation between implicit and explicit attitudes, and the effects of group membership on attitudes and stereotypes.

Overview=======The GNAT (pronounced like the bug) is a flexible technique designed to measure implicit social cognition. Conceptually similar to other implicit measures like the Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, JPSP, 1998), the GNAT assesses automatic associations between concept (e.g., gender) and attribute (e.g., evaluation) categories. The GNAT has two features that distinguish it from other measures of implicit social cognition. First, the GNAT is designed to be use signal detection statistics in its calculation of automatic associations (d-prime), but can also be adapted to utilize response latency as its operational dependent variable. Second, the GNAT is flexible in the establishing of contextual characteristics for the evaluative situation. For example, the IAT requires an attitude toward one category (insects) be assessed relative to a second category (flowers). With the GNAT, experimenters can vary whether insects are evaluated in the context of a single category (flowers), a superordinate category (animals), a generic category (objects) or with no context at all.

This initial research synthesis suggests that the relationship between implicit and explicit attitudes is determined by (at least) two factors – self-presentation and elaboration. A new formulation of the nature of the relationship between implicit and explicit attitudes must be devised to handle these observations. The independent evaluations view does not explain how reliable and predictable relationships can exist between implicit and explicit attitudes. And, the implicit as lie detector view does not anticipate that factors other than self-presentation will moderate IE correspondence. Developing strong theory about the functional relationship between implicit and explicit attitudes requires examination of other factors that could moderate the relationship (e.g., attitude accessibility, attitude importance), and attention to the cognitive processes that give rise to these two modes of evaluation. In other words, the same heavy doses of theorizing about explicit attitudes, if brought to bear on the IE relationship will quickly provide answers to the conditions under which they are likely to be associated or dissociated.

Blanton, Jaccard, Gonzales, and Christie (BJGC, 2006) assert that the Implicit Association Test (IAT) imposes a model that portrays relative preferences as the additive difference between single attitudes. This assertion is misplaced because relative preferences do not necessarily reduce to component attitudes. BJGC also assume that the IAT conditions represent two indicators of the same construct. This assumption is incorrect, and is the cause of their poor-fitting models. The IAT, like other experimental paradigms, contrasts performance between interdependent conditions, and cannot be reduced to component parts. This is true whether calculating a simple difference between conditions, or using the IAT D score. D – an individual effect size that is monotonically related to Cohen's d – codifies the interdependency between IAT conditions. When their unjustified psychometric assumptions are replaced with plausible assumptions, the models fit their data very well, and basis for their poor-fitting models becomes clear.

Legal rights and cultural attitudes towards lesbian women and gay men have shifted rapidly in the early 21st century. Using 683,976 visitors to Project Implicit from February 2006 to August 2013, we investigated whether shifts were also observable in implicit evaluations that occur outside of conscious awareness or control. Similar to public opinion polling, the estimated explicit preference for straight people over lesbian women and gay men was 26% weaker on the last day compared to the first. The estimated implicit preference for straight people declined by 13.4% over the same period. The largest shifts in implicit evaluations occurred among Hispanic, White, female, liberal, and young adult participants; the smallest shifts occurred among Black, Asian, male, conservative, and older adult participants. Societal change in evaluation of lesbian and gay people is not limited to what people are willing and able to report. However, change in implicit evaluation appears to be slower.

Implicit prejudices are social preferences that exist outside of conscious awareness or conscious control. In this review, we summarize evidence for three mechanisms that influence the expression of implicit prejudice: associative change, contextual change, and change in control over implicit prejudice. We then review the evidence (or lack thereof) for answers to five open issues in implicit prejudice reduction research: 1) what shows effectiveness in real-world application; 2) what doesn’t work for implicit prejudice reduction; 3) what interventions produce long-term changes in implicit prejudice; 4) measurement diversity in implicit prejudice reduction research; and 5) the relationship between implicit prejudice and behavior. Addressing these issues provide an agenda for clarifying the conditions and implications of reducing implicit prejudice.

About 70% of more than half a million Implicit Association Tests completed by citizens of 34 countries revealed expected implicit stereotypes associating science with males more than with females. We discovered that nation-level implicit stereotypes predicted nation-level sex differences in 8th-grade science and mathematics achievement. Self-reported stereotypes did not provide additional predictive validity of the achievement gap. We suggest that implicit stereotypes and sex differences in science participation and performance are mutually reinforcing, contributing to the persistent gender gap in science engagement.

A brief version of the Implicit Association Test (BIAT) has been introduced. The present research identified analytical best practices for overall psychometric performance of the BIAT. In 7 studies and multiple replications, we investigated analytic practices with several evaluation criteria: sensitivity to detecting known effects and group differences, internal consistency, relations with implicit measures of the same topic, relations with explicit measures of the same topic and other criterion variables, and resistance to an extraneous influence of average response time. The data transformation algorithms D outperformed other approaches. This replicates and extends the strong prior performance of D compared to conventional analytic techniques. We conclude with recommended analytic practices for standard use of the BIAT.

Beginning January 2014, Psychological Science gave authors the opportunity to signal open data and materials if they qualified for badges that accompanied published articles. Before badges, less than 3% of Psychological Science articles reported open data. After badges, 23% reported open data, with an accelerating trend; 39% reported open data in the first half of 2015, an increase of more than an order of magnitude from baseline. There was no change over time in the low rates of data sharing among comparison journals. Moreover, reporting openness does not guarantee openness. When badges were earned, reportedly available data were more likely to be actually available, correct, usable, and complete than when badges were not earned. Open materials also increased to a weaker degree, and there was more variability among comparison journals. Badges are simple, effective signals to promote open practices and improve preservation of data and materials by using independent repositories.

We examined the role of group membership (being female or male), implicit identity with social groups (me=male/female), and math-gender stereotypes (math=male) in predicting implicit math attitudes (math=good) and math identity (math=me). In addition, we investigated the relationship between implicit and explicit preferences and SAT performance. College students demonstrated negativity toward math and science relative to arts and language on implicit measures. Women showed greater implicit negativity toward math than men did, even in a sample that had selected a math-intensive college major. In addition, associations of math with male and the self with gender related to implicit math attitudes, but those relationships were directly opposing for men and women. Stronger math=male associations corresponded with more negative math attitudes for women, but more positive math attitudes for men. Finally, both implicit and explicit math attitudes were predictive of SAT performance. These results point to the opportunities and constraints on personal preferences that derive from membership in social groups.