Search

Systematic reviewers are advised to search trials registers to minimise risk of reporting biases. However, there has been little research on the impact of searching trials registers on the results of meta-analyses. We aimed to evaluate the impact of searching clinical trials registers for systematic reviews of pharmaceutical or non-pharmaceutical interventions. We searched PubMed, Scopus, Science Citation Index and Social Sciences Citation Index, and Education Collection for systematic reviews with meta-analyses indexed from 2 November to 2 December 2020. A random sample of systematic reviews was initially drawn, and for reviews which considered randomised trials eligible for inclusion, which had not searched a trials register, we searched ClinicalTrials.gov, EudraCT, ANZCTR, and the WHO ICTRP search portal for eligible trials. We compared meta-analytic effect estimates before and after including results from additional trials identified. We found additional trials for 63% (63/101) of eligible reviews; however, trials with results that could contribute to a meta-analysis were identified for only 20% (20/101) of the reviews. On average, there was no difference in the meta-analytic effect estimates before versus after adding the new trials. In summary, searching clinical trial registers led to identification of additional trials for many reviews; however, very few trials had results available for inclusion in meta-analyses. Including results from the new trials led to no change in the meta-analytic estimates, on average. Trials registers would be even more valuable to systematic reviewers if more trialists made use of them (i.e., registered their trials and posted results in a timely manner).

Systematic reviews should be reported completely and accurately so that users can interpret and appraise the review findings appropriately. In this commentary, we describe key features of a major update of the Preferred Reporting Items for Systematic reviews and Meta-Analyses statement--PRISMA 2020--and discuss its implications for research synthesis methodologists.

Systematic reviews are fast increasing in prevalence in the toxicology and environmental health literature. However, how well these complex research projects are being conducted and reported is unclear. Since editors have an essential role in ensuring the scientific quality of manuscripts being published in their journals, a workshop was convened where editors, systematic review practitioners, and research quality control experts could discuss what editors can do to ensure the systematic reviews they publish are of sufficient scientific quality. Interventions were explored along four themes: setting standards; reviewing protocols; optimizing editorial workflows; and measuring the effectiveness of editorial interventions. In total, 58 editorial interventions were proposed. Of these, 26 were shortlisted for being potentially effective, and 5 were prioritized as short-term actions that editors could relatively easily take to improve the quality of published systematic reviews. Recent progress in improving systematic reviews is summarized, and outstanding challenges to further progress are highlighted.

A "P" value, or the magnitude or direction of results can influence decisions about whether, when, and how research findings are disseminated. Regardless of whether an entire study or a particular study result is unavailable because investigators considered the results to be unfavorable, bias in a meta-analysis may occur when available results differ systematically from missing results. In this article, we summarize the empirical evidence for various reporting biases that lead to study results being unavailable for inclusion in systematic reviews, with a focus on health research. These biases include publication bias and selective nonreporting bias. We describe processes that systematic reviewers can use to minimize the risk of bias due to missing results in meta-analyses of health research, such as comprehensive searches and prospective approaches to meta-analysis. We also outline methods that have been designed for assessing risk of bias due to missing results in meta-analyses of health research, including using tools to assess selective nonreporting of results, ascertaining qualitative signals that suggest not all studies were identified, and generating funnel plots to identify small-study effects, one cause of which is reporting bias.

We conducted an international cross-sectional survey of biomedical researchers' perspectives on the reproducibility of research. This study builds on a widely cited 2016 survey on reproducibility and provides a biomedical-specific and contemporary perspective on reproducibility. To sample the community, we randomly selected 400 journals indexed in MEDLINE, from which we extracted the author names and emails from all articles published between October 1, 2020 and October 1, 2021. We invited participants to complete an anonymous online survey which collected basic demographic information, perceptions about a reproducibility crisis, perceived causes of irreproducibility of research results, experience conducting reproducibility studies, and knowledge of funding and training for research on reproducibility. A total of 1,924 participants accessed our survey, of which 1,630 provided useable responses (response rate 7% of 23,234). Key findings include that 72% of participants agreed there was a reproducibility crisis in biomedicine, with 27% of participants indicating the crisis was "significant." The leading perceived cause of irreproducibility was a "pressure to publish" with 62% of participants indicating it "always" or "very often" contributes. About half of the participants (54%) had run a replication of their own previously published study while slightly more (57%) had run a replication of another researcher's study. Just 16% of participants indicated their institution had established procedures to enhance the reproducibility of biomedical research and 67% felt their institution valued new research over replication studies. Participants also reported few opportunities to obtain funding to attempt to reproduce a study and 83% perceived it would be harder to do so than to get funding to do a novel study. Our results may be used to guide training and interventions to improve research reproducibility and to monitor rates of reproducibility over time. The findings are also relevant to policy makers and academic leadership looking to create incentives and research cultures that support reproducibility and value research quality. There is growing interest in the reproducibility of research and ways to enhance transparency. This study uses an international survey of more than 1900 biomedical researchers to reveal the perceived causes of irreproducibility of research results, experience of conducting replication studies, and knowledge of relevant funding and training. [ABSTRACT FROM AUTHOR]

Aim: Previous research has shown patients and the public in Australia generally support medical researchers in making de‐identified research data available to other scientists. However, this research has focussed on certain types of data and recipients. We surveyed Australians affected by cancer to characterize their attitudes toward the sharing of research data with multiple third parties, including the public. Methods: A short, anonymous online survey of Australians with a previous diagnosis of cancer was advertised between October 27, 2022, and February 27, 2023. Quantitative responses were analyzed with descriptive statistics. Free‐text responses were coded deductively and summarised using content analysis. Results: In total, 551 respondents contributed data to the survey. There was strong support for cancer researchers sharing non‐human and de‐identified human research data with clinicians (90% and 95%, respectively) and non‐profit researchers (both 94%). However, fewer participants supported sharing data with for‐profit researchers (both 64%) or publicly (both 61%). When asked if they would hypothetically consent to researchers at their treatment location using and sharing their de‐identified data publicly, only half agreed. In contrast, after being shown a visual representation of the de‐identified survey data, 80% of respondents supported sharing it publicly. Conclusion: Australians affected by cancer support the sharing of research data, particularly with clinicians and non‐profit researchers. Our results also imply that visualization of the data to be shared may enhance support for making it publicly available. These results should help alleviate any concerns about research participants' attitudes toward data sharing, as well as boost researchers' motivation for sharing. [ABSTRACT FROM AUTHOR]

Systematic reviews should provide an unbiased summary of evidence on both benefits and harms. Authors may spin results, particularly harms, whether intentionally or unintentionally. This paper provides a framework for identifying spin and examples of and how to rectify spin. "Spin" refers to misleading reporting, interpretation, and extrapolation of findings in primary and secondary research (such as in systematic reviews). The study of spin primarily focuses on beneficial outcomes. The objectives of this research were threefold: first, to develop a framework for identifying spin associated with harms in systematic reviews of interventions; second, to apply the framework to a set of reviews, thereby pinpointing instances where spin may be present; and finally, to revise the spin examples, offering guidance on how spin can be rectified. The authors developed their framework through an iterative process that engaged an international group of researchers specializing in spin and reporting bias. The framework comprises 12 specific types of spin for harms, grouped by 7 categories across the 3 domains (reporting, interpretation, and extrapolation). The authors subsequently gathered instances of spin from a random sample of 100 systematic reviews of interventions. Of the 58 reviews that assessed harm and the 42 that did not, they found that 28 (48%) and 6 (14%), respectively, had at least 1 of the 12 types of spin for harms. Inappropriate extrapolation of the results and conclusions for harms to populations, interventions, outcomes, or settings not assessed in a review was the most common category of spin in 17 of 100 reviews. The authors revised the examples to remove spin, taking into consideration the context (for example, medical discipline, source population), findings for harms, and methodological limitations of the original reviews. They provide guidance for authors, peer reviewers, and editors in recognizing and rectifying or (preferably) avoiding spin, ultimately enhancing the clarity and accuracy of harms reporting in systematic review publications. [ABSTRACT FROM AUTHOR]

Awards can propel academic careers. They also reflect the culture and values of the scientific community. But do awards incentivize greater transparency, inclusivity, and openness in science? Our cross-disciplinary survey of 222 awards for the "best" journal articles across all 27 SCImago subject areas revealed that journals and learned societies administering such awards generally publish little detail on their procedures and criteria. Award descriptions were brief, rarely including contact details or information on the nominations pool. Nominations of underrepresented groups were not explicitly encouraged, and concepts that align with Open Science were almost absent from the assessment criteria. At the same time, 10% of awards, especially the recently established ones, tended to use article-level impact metrics. USA-affiliated researchers dominated the winner's pool (48%), while researchers from the Global South were uncommon (11%). Sixty-one percent of individual winners were men. Overall, Best Paper awards miss the global calls for greater transparency and equitable access to academic recognition. We provide concrete and implementable recommendations for scientific awards to improve the scientific recognition system and incentives for better scientific practice. Research awards are an integral part of the universal "prestige economy" in science, but do they incentivize greater transparency, inclusivity, and openness? This study uses cross-disciplinary data to explore the level of transparency of publicly available award descriptions and assessment criteria, asking whether such awards contribute to or propagate existing reproducibility crises and inequities in science. [ABSTRACT FROM AUTHOR]

We aimed to explore, in a sample of systematic reviews (SRs) with meta‐analyses of the association between food/diet and health‐related outcomes, whether systematic reviewers selectively included study effect estimates in meta‐analyses when multiple effect estimates were available. We randomly selected SRs of food/diet and health‐related outcomes published between January 2018 and June 2019. We selected the first presented meta‐analysis in each review (index meta‐analysis), and extracted from study reports all study effect estimates that were eligible for inclusion in the meta‐analysis. We calculated the Potential Bias Index (PBI) to quantify and test for evidence of selective inclusion. The PBI ranges from 0 to 1; values above or below 0.5 suggest selective inclusion of effect estimates more or less favourable to the intervention, respectively. We also compared the index meta‐analytic estimate to the median of a randomly constructed distribution of meta‐analytic estimates (i.e., the estimate expected when there is no selective inclusion). Thirty‐nine SRs with 312 studies were included. The estimated PBI was 0.49 (95% CI 0.42–0.55), suggesting that the selection of study effect estimates from those reported was consistent with a process of random selection. In addition, the index meta‐analytic effect estimates were similar, on average, to what we would expect to see in meta‐analyses generated when there was no selective inclusion. Despite this, we recommend that systematic reviewers report the methods used to select effect estimates to include in meta‐analyses, which can help readers understand the risk of selective inclusion bias in the SRs. [ABSTRACT FROM AUTHOR]

Systematic reviews often encounter primary studies that report multiple effect sizes based on data from the same participants. These have the potential to introduce statistical dependency into the meta-analytic data set. In this paper, we provide a tutorial on dealing with effect size multiplicity within studies in the context of meta-analyses of intervention and association studies, recommending a three-step approach. The first step is to define the research question and consider the extent to which it mainly reflects interest in mean effect sizes (which we term a "convergent" approach) or an interest in exploring heterogeneity (which we term a "divergent" approach). A second step is to identify the types of multiplicities that appear in the initial database of effect sizes relevant to the research question, and we propose a categorization scheme to differentiate them. The third step is to select a strategy for dealing with each type of multiplicity. The researcher can choose between a "reductionist" meta-analytic approach, which is characterized by inclusion of a single effect size per study, and an "integrative" approach, characterized by inclusion of multiple effect sizes per study. We present an overview of available analysis strategies for dealing with effect size multiplicity within studies and provide recommendations intended to help researchers decide which strategy might be preferable in particular situations. Last, we offer caveats and cautions about addressing the challenges multiplicity poses for systematic reviews and meta-analyses.