Search

Journal editors have a large amount of power to advance open science in their respective fields by incentivising and mandating open policies and practices at their journals. The Data PASS Journal Editors Discussion Interface (JEDI, an online community for social science journal editors: www.dpjedi.org ) has collated several resources on embedding open science in journal editing ( www.dpjedi.org/resources ). However, it can be overwhelming as an editor new to open science practices to know where to start. For this reason, we created a guide for journal editors on how to get started with open science. The guide outlines steps that editors can take to implement open policies and practices within their journal, and goes through the what, why, how, and worries of each policy and practice. This manuscript introduces and summarizes the guide (full guide: https://doi.org/10.31219/osf.io/hstcx ).

Clearinghouses are influential repositories of information on the effectiveness of social interventions. To identify which interventions are "evidence-based," clearinghouses review intervention evaluations using published standards of evidence that focus primarily on internal validity and causal inferences. Open science practices can improve trust in evidence from evaluations on the effectiveness of social interventions. Including open science practices in clearinghouse standards of evidence is one of many efforts that could increase confidence in designations of interventions as "evidence-based." In this study, we examined the policies, procedures, and practices of 10 federal evidence clearinghouses that review preventive interventions--an important and influential subset of all evidence clearinghouses. We found that seven consider at least one open science practice when evaluating interventions: replication (6 of 10 clearinghouses), public availability of results (6), investigator conflicts of interest (3), design and analysis transparency (3), study registration (2), and protocol sharing (1). We did not identify any policies, procedures, or practices related to analysis plan registration, data sharing, code sharing, material sharing, and citation standards. We provide a framework with specific recommendations to help federal and other evidence clearinghouses implement the Transparency and Openness Promotion (TOP) Guidelines. Our proposed "TOP Guidelines for Clearinghouses" includes reporting whether evaluations used open science practices, incorporating open science practices in their standards for receiving "evidence-based" designations, and verifying that evaluations used open science practices. Doing so could increase the trustworthiness of evidence used for policy making and support improvements throughout the evidence ecosystem.

Systematic reviews should provide an unbiased summary of evidence on both benefits and harms. Authors may spin results, particularly harms, whether intentionally or unintentionally. This paper provides a framework for identifying spin and examples of and how to rectify spin. "Spin" refers to misleading reporting, interpretation, and extrapolation of findings in primary and secondary research (such as in systematic reviews). The study of spin primarily focuses on beneficial outcomes. The objectives of this research were threefold: first, to develop a framework for identifying spin associated with harms in systematic reviews of interventions; second, to apply the framework to a set of reviews, thereby pinpointing instances where spin may be present; and finally, to revise the spin examples, offering guidance on how spin can be rectified. The authors developed their framework through an iterative process that engaged an international group of researchers specializing in spin and reporting bias. The framework comprises 12 specific types of spin for harms, grouped by 7 categories across the 3 domains (reporting, interpretation, and extrapolation). The authors subsequently gathered instances of spin from a random sample of 100 systematic reviews of interventions. Of the 58 reviews that assessed harm and the 42 that did not, they found that 28 (48%) and 6 (14%), respectively, had at least 1 of the 12 types of spin for harms. Inappropriate extrapolation of the results and conclusions for harms to populations, interventions, outcomes, or settings not assessed in a review was the most common category of spin in 17 of 100 reviews. The authors revised the examples to remove spin, taking into consideration the context (for example, medical discipline, source population), findings for harms, and methodological limitations of the original reviews. They provide guidance for authors, peer reviewers, and editors in recognizing and rectifying or (preferably) avoiding spin, ultimately enhancing the clarity and accuracy of harms reporting in systematic review publications. [ABSTRACT FROM AUTHOR]

Data for individual trials included in systematic reviews may be available in multiple sources. For example, a single trial might be reported in 2 journal articles and 3 conference abstracts. Because of differences across sources, source selection can influence the results of systematic reviews. We used our experience in the Multiple Data Sources in Systematic Reviews (MUDS) study, and evidence from previous studies, to develop practical guidance for using multiple data sources in systematic reviews. We recommend the following: (1) Specify which sources you will use. Before beginning a systematic review, consider which sources are likely to contain the most useful data. Try to identify all relevant reports and to extract information from the most reliable sources. (2) Link individual trials with multiple sources. Write to authors to determine which sources are likely related to the same trials. Use a modified Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flowchart to document both the selection of trials and the selection of sources. (3) Follow a prespecified protocol for extracting trial characteristics from multiple sources. Identify differences among sources, and contact study authors to resolve differences if possible. (4) Prespecify outcomes and results to examine in the review and meta-analysis. In your protocol, describe how you will handle multiple outcomes within each domain of interest. Look for outcomes using all eligible sources. (5) Identify which data sources were included in the review. Consider whether the results might have been influenced by data sources used. (6) To reduce bias, and to reduce research waste, share the data used in your review. [This article was written for the MUDS investigators.]

This article is based on a presentation that was made at the 2014 annual meeting of the editorial board of "Health Education & Behavior." The article addresses critical issues related to standards of scientific reporting in journals, including concerns about external and internal validity and reporting bias. It reviews current reporting guidelines, effects of adopting guidelines, and offers suggestions for improving reporting. The evidence about the effects of guideline adoption and implementation is briefly reviewed. Recommendations for adoption and implementation of appropriate guidelines, including considerations for journals, are provided.

Objective: Trials often may report several similar outcomes measured on different test instruments. We explored a method for synthesising treatment effect information both within and between trials and for reporting treatment effects on a common scale as an alternative to standardisation Study design: We applied a procedure that simultaneously estimates a pooled treatment effect and the "mapping" ratios between the treatment effects on test instruments in a connected network. Standardised and non-standardised treatment effects were compared. The methods were illustrated in a dataset of 22 trials of selective serotonin reuptake inhibitors against placebo for social anxiety disorder, each reporting treatment effects on between one and six of a total nine test instruments. Results: Ratios of treatment effects on different test instruments varied from trial to trial, with a coefficient of variation of 18% (95% credible interval 11-29%). Standardised effect models fitted the data less well, and standardised treatment effects were estimated with less relative precision than non-standardised effects and with greater relative heterogeneity. Conclusion: Simultaneous synthesis of treatment effects and mapping to a common scale make fewer assumptions than standardising by dividing effects by the sample standard deviation, allow results to be reported on a common scale, and deliver estimates with superior relative precision.