Search

This paper explores judgements about the replicability of social and behavioural sciences research and what drives those judgements. Using a mixed methods approach, it draws on qualitative and quantitative data elicited from groups using a structured approach called the IDEA protocol ('investigate', 'discuss', 'estimate' and 'aggregate'). Five groups of five people with relevant domain expertise evaluated 25 research claims that were subject to at least one replication study. Participants assessed the probability that each of the 25 research claims would replicate (i.e. that a replication study would find a statistically significant result in the same direction as the original study) and described the reasoning behind those judgements. We quantitatively analysed possible correlates of predictive accuracy, including self-rated expertise and updating of judgements after feedback and discussion. We qualitatively analysed the reasoning data to explore the cues, heuristics and patterns of reasoning used by participants. Participants achieved 84% classification accuracy in predicting replicability. Those who engaged in a greater breadth of reasoning provided more accurate replicability judgements. Some reasons were more commonly invoked by more accurate participants, such as 'effect size' and 'reputation' (e.g. of the field of research). There was also some evidence of a relationship between statistical literacy and accuracy.

This is the project page for a systematic review and individual participant data meta-analysis that aims to synthesise the findings of medical and health sciences studies that have empirically investigated the prevalence of data and code sharing.

This paper explores judgements about the replicability of social and behavioural sciences research and what drives those judgements. Using a mixed methods approach, it draws on qualitative and quantitative data elicited from groups using a structured approach called the IDEA protocol (‘investigate’, ‘discuss’, ‘estimate’ and ‘aggregate’). Five groups of five people with relevant domain expertise evaluated 25 research claims that were subject to at least one replication study. Participants assessed the probability that each of the 25 research claims would replicate (i.e. that a replication study would find a statistically significant result in the same direction as the original study) and described the reasoning behind those judgements. We quantitatively analysed possible correlates of predictive accuracy, including self-rated expertise and updating of judgements after feedback and discussion. We qualitatively analysed the reasoning data to explore the cues, heuristics and patterns of reasoning used by participants. Participants achieved 84% classification accuracy in predicting replicability. Those who engaged in a greater breadth of reasoning provided more accurate replicability judgements. Some reasons were more commonly invoked by more accurate participants, such as ‘effect size’ and ‘reputation’ (e.g. of the field of research). There was also some evidence of a relationship between statistical literacy and accuracy.

Today, the four Atlantic continents – North and South America, Africa, and Europe – are connecting as never before. With little fanfare, the Atlantic Basin is becoming a central arena of globalization and a microcosm of key global trends, including the diffusion of power, deepening interdependencies, and spreading transnational risks. It has rapidly become one of the world’s principal energy reservoirs. It is the world’s most heavily traveled ocean and has become the inland sea to the vast majority of the world’s democracies. The Atlantic data seaway, already the busiest in the world, is building out fast. Pan-Atlantic commercial flows rival, and in such areas as services, investment and digital commerce exceed, those of the Pacific. Never have so many workers and consumers entered the Atlantic economy as quickly as in the past two decades. The Atlantic Ocean plays a pivotal role with respect to changing global climate and weather patterns, and offers the most immediate opportunities for “blue growth” strategies to harvest its riches. Yet the Atlantic Hemisphere is a region of extreme wealth and poverty. Atlantic peoples are on the front lines of global climate change, greater superstorms, and rising sea levels. Together they are threatened by a growing pan-Atlantic nexus of drugs, guns, and terror. Despite these dynamics, pan-Atlantic governance mechanisms and diplomatic cooperation remain in their infancy. Twenty Atlantic countries recently pioneered a new initiative to explore greater pan-Atlantic cooperation in areas ranging from sustainable development and climate issues to maritime governance and human security. Potentially transformative digital and energy dynamics are also ripe for pan-Atlantic attention. It is time to consider ways to erase the invisible line dividing the South and North Atlantic, so that societies across this vast space are better able to face the new world rising before us.

Systematic Review - Masters