Search

We conducted an international cross-sectional survey of biomedical researchers' perspectives on the reproducibility of research. This study builds on a widely cited 2016 survey on reproducibility and provides a biomedical-specific and contemporary perspective on reproducibility. To sample the community, we randomly selected 400 journals indexed in MEDLINE, from which we extracted the author names and emails from all articles published between October 1, 2020 and October 1, 2021. We invited participants to complete an anonymous online survey which collected basic demographic information, perceptions about a reproducibility crisis, perceived causes of irreproducibility of research results, experience conducting reproducibility studies, and knowledge of funding and training for research on reproducibility. A total of 1,924 participants accessed our survey, of which 1,630 provided useable responses (response rate 7% of 23,234). Key findings include that 72% of participants agreed there was a reproducibility crisis in biomedicine, with 27% of participants indicating the crisis was "significant." The leading perceived cause of irreproducibility was a "pressure to publish" with 62% of participants indicating it "always" or "very often" contributes. About half of the participants (54%) had run a replication of their own previously published study while slightly more (57%) had run a replication of another researcher's study. Just 16% of participants indicated their institution had established procedures to enhance the reproducibility of biomedical research and 67% felt their institution valued new research over replication studies. Participants also reported few opportunities to obtain funding to attempt to reproduce a study and 83% perceived it would be harder to do so than to get funding to do a novel study. Our results may be used to guide training and interventions to improve research reproducibility and to monitor rates of reproducibility over time. The findings are also relevant to policy makers and academic leadership looking to create incentives and research cultures that support reproducibility and value research quality.

Objective To identify current and emerging self-care interventions to improve maternity healthcare.Design Scoping review.Data sources MEDLINE, Embase, EmCare, PsycINFO, Cochrane CENTRAL/CDSR, CINAHL Plus (last searched on 17 October 2021).Eligibility criteria Evidence syntheses, interventional or observational studies describing any tool, resource or strategy to facilitate self-care in women preparing to get pregnant, currently pregnant, giving birth or post partum.Data extraction/synthesis Screening and data collection were conducted independently by two reviewers. Self-care interventions were identified based on predefined criteria and inductively organised into 11 categories. Characteristics of study design, interventions, participants and outcomes were recorded.Results We identified eligible 580 studies. Many included studies evaluated interventions in high-income countries (45%) and during antenatal care (76%). Self-care categories featuring highest numbers of studies were diet and nutrition (26% of all studies), physical activity (24%), psychosocial strategies (18%) and other lifestyle adjustments (17%). Few studies featured self-care interventions for sexual health and postpartum family planning (2%), self-management of medication (3%) and self-testing/sampling (3%). Several venues to introduce self-care were described: health facilities (44%), community venues (14%), digital platforms (18%), partner/peer support (7%) or over-the-counter products (13%). Involvement of health and community workers were described in 38% and 8% of studies, who supported self-care interventions by providing therapeutics for home use, training or counselling. The most common categories of outcomes evaluated were neonatal outcomes (eg, birth weight) (31%), maternal mental health (26%) and labour outcomes (eg, duration of labour) (22%).Conclusion Self-care interventions in maternal care are diverse in their applications, implementation characteristics and intended outcomes. Many self-care interventions were implemented with support from the health system at initial stages of use and uptake. Some promising self-care interventions require further primary research, though several are matured and up-to-date evidence syntheses are needed. Research on self-care in the preconception period is lacking.

Introduction The Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) reporting guideline establishes a minimum set of items to be reported in any randomised controlled trial (RCT) protocol. The Template for Intervention Description and Replication (TIDieR) reporting guideline was developed to improve the reporting of interventions in RCT protocols and results papers. Reporting completeness in protocols of diet or nutrition-related RCTs has not been systematically investigated. We aim to identify published protocols of diet or nutrition-related RCTs, assess their reporting completeness and identify the main reporting limitations remaining in this field.Methods and analysis We will conduct a meta-research study of RCT protocols published in journals indexed in at least one of six selected databases between 2012 and 2022. We have run a search in PubMed, Embase, CINAHL, Web of Science, PsycINFO and Global Health using a search strategy designed to identify protocols of diet or nutrition-related RCTs. Two reviewers will independently screen the titles and abstracts of records yielded by the search in Rayyan. The full texts will then be read to confirm protocol eligibility. We will collect general study features (publication information, types of participants, interventions, comparators, outcomes and study design) of all eligible published protocols in this contemporary sample. We will assess reporting completeness in a randomly selected sample of them and identify their main reporting limitations. We will compare this subsample with the items in the SPIRIT and TIDieR statements. For all data collection, we will use data extraction forms in REDCap. This protocol is registered on the Open Science Framework (DOI: 10.17605/OSF.IO/YWEVS).Ethics and dissemination This study will undertake a secondary analysis of published data and does not require ethical approval. The results will be disseminated through journals and conferences targeting stakeholders involved in nutrition research.

Background: While the PRISMA 2020 statement is intended to guide the reporting of original systematic reviews, updated systematic reviews, and living systematic reviews (LSRs), its explanation and elaboration document notes that additional considerations for updated systematic reviews and LSRs may need to be addressed. This paper reports the protocol for developing an extension of the PRISMA 2020 statement for LSRs. Methods: We will follow the EQUATOR Network’s guidance for developing health research reporting guidelines. We will review the literature to identify possible items of the PRISMA 2020 checklist that need modification, as well as new items that need to be added. Then, we will survey representatives of different stakeholder groups for their views on the proposed modifications of the PRISMA 2020 checklist. We will summarize, present, and discuss the results of the survey in an online meeting, aiming to reach consensus on the content of the LSR extension. We will then draft the checklist, explanation and elaboration for each item, and flow diagram for the PRISMA 2020 extension. Then, we will share these initial documents with stakeholder representatives for final feedback and approval. Discussion: We anticipate that the PRISMA 2020 extension for LSRs will benefit LSR authors, editors, and peer reviewers of LSRs, as well as different users of LSRs, including guideline developers, policy makers, healthcare providers, patients, and other stakeholders.

Introduction Health services generate large amounts of routine health data (eg, administrative databases, disease registries and electronic health records), which have important secondary uses for research. Increases in the availability and the ability to access and analyse large amounts of data represent a major opportunity for conducting studies on the possible relationships between complex diseases. The objective of this study will be to evaluate the design, methods and reporting of studies conducted using observational routinely collected health data for investigating the link between cancer and neurodegenerative diseases.Methods and analysis This is the protocol for a meta-research study. We registered the study protocol within the Open Science Framework: https://osf.io/h2qjg. We will evaluate observational studies (eg, cohort and case–control) conducted using routinely collected health data for investigating the associations between cancer and neurodegenerative diseases (such as Alzheimer’s disease, amyotrophic lateral sclerosis/motor neuron disease, Huntington’s disease, multiple sclerosis and Parkinson’s disease). The following electronic databases will be searched (from their inception onwards): MEDLINE, Embase and Web of Science Core Collection. Screening and selection of articles will be conducted by at least two researchers. Potential discrepancies will be resolved via discussion. Design, methods and reporting characteristics in each article will be extracted using a standardised data extraction form. Information on general, methodological and transparency items will be reported. We will summarise our findings with tables and graphs (eg, bar charts, forest plots).Ethics and dissemination Due to the nature of the proposed study, no ethical approval will be required. We plan to publish the full study in an open access peer-reviewed journal and disseminate the findings at scientific conferences and via social media. All data will be deposited in a cross-disciplinary public repository.

Background: While the PRISMA 2020 statement is intended to guide the reporting of original systematic reviews, updated systematic reviews, and living systematic reviews (LSRs), its explanation and elaboration document notes that additional considerations for updated systematic reviews and LSRs may need to be addressed. This paper reports the protocol for developing an extension of the PRISMA 2020 statement for LSRs. Methods: We will follow the EQUATOR Network’s guidance for developing health research reporting guidelines. We will review the literature to identify possible items of the PRISMA 2020 checklist that need modification, as well as new items that need to be added. Then, we will survey representatives of different stakeholder groups for their views on the proposed modifications of the PRISMA 2020 checklist. We will summarize, present, and discuss the results of the survey in an online meeting, aiming to reach consensus on the content of the LSR extension. We will then draft the checklist, explanation and elaboration for each item, and flow diagram for the PRISMA 2020 extension. Then, we will share these initial documents with stakeholder representatives for final feedback and approval. Discussion: We anticipate that the PRISMA 2020 extension for LSRs will benefit LSR authors, editors, and peer reviewers of LSRs, as well as different users of LSRs, including guideline developers, policy makers, healthcare providers, patients, and other stakeholders.

Individual participant data meta-analyses enable detailed checking of data quality and more complex analyses than standard study-level synthesis of summary data based on publications. However, there is limited existing guidance on the specific systematic checks that should be undertaken to confirm and enhance data quality for individual participant data meta-analyses and how to conduct these checks. We aim to address this gap by developing a checklist of items for data quality checking and cleaning to be applied to individual participant data meta-analyses of randomised trials. This study will comprise three phases: 1) a scoping review to identify potential checklist items; 2) two e-Delphi survey rounds among an invited panel of experts followed by a consensus meeting; and 3) pilot testing and refinement of the checklist, including development of an accompanying R-markdown program to facilitate its uptake.

Objective To compare results reporting and the presence of spin in COVID-19 study preprints with their finalised journal publications.Design Cross-sectional study.Setting International medical literature.Participants Preprints and final journal publications of 67 interventional and observational studies of COVID-19 treatment or prevention from the Cochrane COVID-19 Study Register published between 1 March 2020 and 30 October 2020.Main outcome measures Study characteristics and discrepancies in (1) results reporting (number of outcomes, outcome descriptor, measure, metric, assessment time point, data reported, reported statistical significance of result, type of statistical analysis, subgroup analyses (if any), whether outcome was identified as primary or secondary) and (2) spin (reporting practices that distort the interpretation of results so they are viewed more favourably).Results Of 67 included studies, 23 (34%) had no discrepancies in results reporting between preprints and journal publications. Fifteen (22%) studies had at least one outcome that was included in the journal publication, but not the preprint; eight (12%) had at least one outcome that was reported in the preprint only. For outcomes that were reported in both preprints and journals, common discrepancies were differences in numerical values and statistical significance, additional statistical tests and subgroup analyses and longer follow-up times for outcome assessment in journal publications.At least one instance of spin occurred in both preprints and journals in 23/67 (34%) studies, the preprint only in 5 (7%), and the journal publications only in 2 (3%). Spin was removed between the preprint and journal publication in 5/67 (7%) studies; but added in 1/67 (1%) study.Conclusions The COVID-19 preprints and their subsequent journal publications were largely similar in reporting of study characteristics, outcomes and spin. All COVID-19 studies published as preprints and journal publications should be critically evaluated for discrepancies and spin.

We aimed to explore, in a sample of systematic reviews (SRs) with meta-analyses of the association between food/diet and health-related outcomes, whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available. We randomly selected SRs of food/diet and health-related outcomes published between January 2018 and June 2019. We selected the first presented meta-analysis in each review (index meta-analysis), and extracted from study reports all study effect estimates that were eligible for inclusion in the meta-analysis. We calculated the Potential Bias Index (PBI) to quantify and test for evidence of selective inclusion. The PBI ranges from 0 to 1; values above or below 0.5 suggest selective inclusion of effect estimates more or less favourable to the intervention, respectively. We also compared the index meta-analytic estimate to the median of a randomly constructed distribution of meta-analytic estimates (i.e., the estimate expected when there is no selective inclusion). Thirty-nine SRs with 312 studies were included. The estimated PBI was 0.49 (95% CI 0.42-0.55), suggesting that the selection of study effect estimates from those reported was consistent with a process of random selection. In addition, the index meta-analytic effect estimates were similar, on average, to what we would expect to see in meta-analyses generated when there was no selective inclusion. Despite this, we recommend that systematic reviewers report the methods used to select effect estimates to include in meta-analyses, which can help readers understand the risk of selective inclusion bias in the SRs.

Background: Interrupted time series (ITS) studies contribute importantly to systematic reviews of population-level interventions. We aimed to develop and validate search filters to retrieve ITS studies in MEDLINE and PubMed. Methods: A total of 1017 known ITS studies (published 2013-2017) were analysed using text mining to generate candidate terms. A control set of 1398 time-series studies were used to select differentiating terms. Various combinations of candidate terms were iteratively tested to generate three search filters. An independent set of 700 ITS studies was used to validate the filters' sensitivities. The filters were test-run in Ovid MEDLINE and the records randomly screened for ITS studies to determine their precision. Finally, all MEDLINE filters were translated to PubMed format and their sensitivities in PubMed were estimated. Results: Three search filters were created in MEDLINE: a "precision-maximising" filter with high precision (78%; 95% CI 74%-82%) but moderate sensitivity (63%; 59%-66%), most appropriate when there are limited resources to screen studies; a sensitivity-and-precision-maximising filter with higher sensitivity (81%; 77%-83%) but lower precision (32%; 28%-36%), providing a balance between expediency and comprehensiveness; and a "sensitivity-maximising" filter with high sensitivity (88%; 85%-90%) but likely very low precision, useful when combined with specific content terms. Similar sensitivity estimates were found for PubMed versions. Conclusion: Our filters strike different balances between comprehensiveness and screening workload and suit different research needs. Retrieval of ITS studies would be improved if authors identified the ITS design in the titles.

In an effort to better utilize published evidence obtained from animal experiments, systematic reviews of preclinical studies are increasingly more common-along with the methods and tools to appraise them (e.g., SYstematic Review Center for Laboratory animal Experimentation [SYRCLE's] risk of bias tool). We performed a cross-sectional study of a sample of recent preclinical systematic reviews (2015-2018) and examined a range of epidemiological characteristics and used a 46-item checklist to assess reporting details. We identified 442 reviews published across 43 countries in 23 different disease domains that used 26 animal species. Reporting of key details to ensure transparency and reproducibility was inconsistent across reviews and within article sections. Items were most completely reported in the title, introduction, and results sections of the reviews, while least reported in the methods and discussion sections. Less than half of reviews reported that a risk of bias assessment for internal and external validity was undertaken, and none reported methods for evaluating construct validity. Our results demonstrate that a considerable number of preclinical systematic reviews investigating diverse topics have been conducted; however, their quality of reporting is inconsistent. Our study provides the justification and evidence to inform the development of guidelines for conducting and reporting preclinical systematic reviews.

Matthew Page and co-authors describe PRISMA 2020, an updated reporting guideline for systematic reviews and meta-analyses.

BackgroundEffective drug treatments for Covid-19 are needed to decrease morbidity and mortality for the individual and to alleviate pressure on health care systems. Remdesivir showed promising results in early randomised trials but subsequently a large publicly funded trial has shown less favourable results and the evidence is interpreted differently in clinical guidelines. Systematic reviews of remdesivir have been published, but none have systematically searched for unpublished data, including regulatory documents, and assessed the risk of bias due to missing evidence.MethodsWe will conduct a systematic review of randomised trials comparing remdesivir to placebo or standard of care in any setting. We will include trials regardless of the severity of disease and we will include trials examining remdesivir for indications other than Covid-19 for harms analyses. We will search websites of regulatory agencies, trial registries, bibliographic databases, preprint servers and contact trial sponsors to obtain all available data, including unpublished clinical data, for all eligible trials. Our primary outcomes will be all-cause mortality and serious adverse events. Our secondary outcomes will be length of hospital stay, time to death, severe disease, and adverse events. We will assess the risk of bias using the Cochranes Risk of Bias 2 tool and the risk of bias due to missing evidence (e.g. publication bias, selective reporting bias) using the ROB-ME tool. Where appropriate we will synthesise study results by conducting random-effects meta-analysis. We will present our findings in a Summary of Findings table and rate the certainty of the evidence using the GRADE approach.DiscussionBy conducting a comprehensive systematic review including unpublished data (where available), we expect to be able to provide valuable information for patients and clinicians about the benefits and harms of remdesivir for the treatment of Covid-19. This will help to ensure optimal treatment for individual patients and optimal utilisation of health care resources.Systematic review registrationCRD42021255915.

Introduction Current recommendations for vitamin D and calcium in dietary guidelines and bone health guidelines vary significantly among countries and professional organisations. It is unknown whether the methods used to develop these recommendations followed a rigourous process and how the differences in methods used may affect the recommended intakes of vitamin D and calcium. The objectives of this study are (1) collate and compare recommendations for vitamin D and calcium across guidelines, (2) appraise methodological quality of the guideline recommendations and (3) identify methodological factors that may affect the recommended intakes for vitamin D and calcium. This study will make a significant contribution to enhancing the methodological rigour in public health guidelines for vitamin D and calcium recommendations.Methods and analyses We will conduct a systematic review to evaluate vitamin D and calcium recommendations for osteoporosis prevention in generally healthy middle-aged and older adults. Methodological assessment will be performed for each guideline against those outlined in the 2014 WHO handbook for guideline development. A systematic search strategy will be applied to locate food-based dietary guidelines and bone health guidelines indexed in various electronic databases, guideline repositories and grey literature from 1 January 2009 to 28 February 2019. Descriptive statistics will be used to summarise the data on intake recommendation and on proportion of guidelines consistent with the WHO criteria. Logistic regression, if feasible, will be used to assess the relationships between the methodological factors and the recommendation intakes.Ethics and dissemination Ethics approval is not required as we will only extract published data or information from the published guidelines. Results of this review will be disseminated through conference presentations and peer-reviewed publications.PROSPERO registration number CRD42019126452

The management of comorbidity and multimorbidity poses major challenges to health services around the world. Analysis of scientific research in comorbidity and multimorbidity is limited in the biomedical literature. This study aimed to map global scientific research in comorbidity and multimorbidity to understand the maturity and growth of the area during the past decades.This was a cross-sectional analysis of the Web of Science. Searches were run from inception until November 8, 2016. We included research articles or reviews with no restrictions by language or publication date. Data abstraction was done by one researcher. A process of standardization was conducted by two researchers to unify different terms and grammatical variants and to remove typographical, transcription, and/or indexing errors. All potential discrepancies were resolved via discussion. Descriptive analyses were conducted (including the number of papers, citations, signatures, most prolific authors, countries, journals and keywords). Network analyses of collaborations between countries and co-words were presented. During the period 1970-2016, 85994 papers (64.0% in 2010-2016) were published in 3500 journals. There was wide diversity in the specialty of the journals, with psychiatry (16558 papers; 19.3%), surgery (9570 papers; 11.1%), clinical neurology (9275 papers; 10.8%), and general and internal medicine (7622 papers; 8.9%) the most common. PLOS One (1223 papers; 1.4%), the Journal of Affective Disorders (1154 papers; 1.3%), the Journal of Clinical Psychiatry (727 papers; 0.8%), the Journal of the American Geriatrics Society (634 papers; 0.7%) and Obesity Surgery (588 papers; 0.7%) published the largest number of papers. 168 countries were involved in the production of papers. The global productivity ranking was headed by the United States (37624 papers), followed by the United Kingdom (7355 papers), Germany (6899 papers) and Canada (5706 papers). Twenty authors who published 100 or more papers were identified; the most prolific authors were affiliated with Harvard Medical School, State University of New York Upstate Medical University, National Taiwan Normal University and China Medical University. The 50 most cited papers ("citation classics" with at least 1000 citations) were published in 20 journals, led by JAMA Psychiatry (11 papers) and JAMA (10 papers). The most cited papers provided contributions focusing on methodological aspects (e.g. Charlson Comorbidity Index, Elixhauser Comorbidity Index, APACHE prognostic system), but also important studies on chronic diseases (e.g. epidemiology of mental disorders and its correlates by the U.S. National Comorbidity Survey, Fried's frailty phenotype or the management of obesity).Ours is the first analysis of global scientific research in comorbidity and multimorbidity. Scientific production in the field is increasing worldwide with research leadership of Western countries, most notably, the United States.

Attention deficit hyperactivity disorder (ADHD) is one of the most commonly diagnosed psychiatric disorders in childhood. A wide variety of treatments have been used for the management of ADHD. We aimed to compare the efficacy and safety of pharmacological, psychological and complementary and alternative medicine interventions for the treatment of ADHD in children and adolescents.We performed a systematic review with network meta-analyses. Randomised controlled trials (≥ 3 weeks follow-up) were identified from published and unpublished sources through searches in PubMed and the Cochrane Library (up to April 7, 2016). Interventions of interest were pharmacological (stimulants, non-stimulants, antidepressants, antipsychotics, and other unlicensed drugs), psychological (behavioural, cognitive training and neurofeedback) and complementary and alternative medicine (dietary therapy, fatty acids, amino acids, minerals, herbal therapy, homeopathy, and physical activity). The primary outcomes were efficacy (treatment response) and acceptability (all-cause discontinuation). Secondary outcomes included discontinuation due to adverse events (tolerability), as well as serious adverse events and specific adverse events. Random-effects Bayesian network meta-analyses were conducted to obtain estimates as odds ratios (ORs) with 95% credibility intervals. We analysed interventions by class and individually. 190 randomised trials (52 different interventions grouped in 32 therapeutic classes) that enrolled 26114 participants with ADHD were included in complex networks. At the class level, behavioural therapy (alone or in combination with stimulants), stimulants, and non-stimulant seemed significantly more efficacious than placebo. Behavioural therapy in combination with stimulants seemed superior to stimulants or non-stimulants. Stimulants seemed superior to behavioural therapy, cognitive training and non-stimulants. Behavioural therapy, stimulants and their combination showed the best profile of acceptability. Stimulants and non-stimulants seemed well tolerated. Among medications, methylphenidate, amphetamine, atomoxetine, guanfacine and clonidine seemed significantly more efficacious than placebo. Methylphenidate and amphetamine seemed more efficacious than atomoxetine and guanfacine. Methylphenidate and clonidine seemed better accepted than placebo and atomoxetine. Most of the efficacious pharmacological treatments were associated with harms (anorexia, weight loss and insomnia), but an increased risk of serious adverse events was not observed. There is lack of evidence for cognitive training, neurofeedback, antidepressants, antipsychotics, dietary therapy, fatty acids, and other complementary and alternative medicine. Overall findings were limited by the clinical and methodological heterogeneity, small sample sizes of trials, short-term follow-up, and the absence of high-quality evidence; consequently, results should be interpreted with caution.Clinical differences may exist between the pharmacological and non-pharmacological treatment used for the management of ADHD. Uncertainties about therapies and the balance between benefits, costs and potential harms should be considered before starting treatment. There is an urgent need for high-quality randomised trials of the multiple treatments for ADHD in children and adolescents. PROSPERO, number CRD42014015008.

Background Interrupted time-series (ITS) studies are commonly used to examine the effects of interventions targeted at populations. Suppression of ITS studies or results within these studies, known as reporting bias, has the potential to bias the evidence-base on a particular topic, with potential consequences for healthcare decision-making. Therefore, we aim to determine whether there is evidence of reporting bias among ITS studies. Methods We will conduct a search for published protocols of ITS studies and reports of their results in PubMed, MEDLINE, and Embase up to December 31, 2022. We contact the authors of the ITS studies to seek information about their study, including submission status, data for unpublished results, and reasons for non-publication or non-reporting of certain outcomes. We will examine if there is evidence of publication bias by examining whether time-to-publication is influenced by the statistical significance of the study’s results for the primary research question using Cox proportional hazards regression. We will examine whether there is evidence of discrepancies in outcomes by comparing those specified in the protocols with those in the reports of results, and we will examine whether the statistical significance of an outcome’s result is associated with how completely that result is reported using multivariable logistic regression. Finally, we will examine discrepancies between protocols and reports of results in the methods by examining the data collection processes, model characteristics, and statistical analysis methods. Discrepancies will be summarized using descriptive statistics. Discussion These findings will inform systematic reviewers and policymakers about the extent of reporting biases and may inform the development of mechanisms to reduce such biases.

BackgroundSystematic reviews (SRs) can help decision makers interpret the deluge of published biomedical literature. However, a SR may be of limited use if the methods used to conduct the SR are flawed, and reporting of the SR is incomplete. To our knowledge, since 2004 there has been no cross-sectional study of the prevalence, focus, and completeness of reporting of SRs across different specialties. Therefore, the aim of our study was to investigate the epidemiological and reporting characteristics of a more recent cross-section of SRs.Methods and findingsWe searched MEDLINE to identify potentially eligible SRs indexed during the month of February 2014. Citations were screened using prespecified eligibility criteria. Epidemiological and reporting characteristics of a random sample of 300 SRs were extracted by one reviewer, with a 10% sample extracted in duplicate. We compared characteristics of Cochrane versus non-Cochrane reviews, and the 2014 sample of SRs versus a 2004 sample of SRs. We identified 682 SRs, suggesting that more than 8,000 SRs are being indexed in MEDLINE annually, corresponding to a 3-fold increase over the last decade. The majority of SRs addressed a therapeutic question and were conducted by authors based in China, the UK, or the US; they included a median of 15 studies involving 2,072 participants. Meta-analysis was performed in 63% of SRs, mostly using standard pairwise methods. Study risk of bias/quality assessment was performed in 70% of SRs but was rarely incorporated into the analysis (16%). Few SRs (7%) searched sources of unpublished data, and the risk of publication bias was considered in less than half of SRs. Reporting quality was highly variable; at least a third of SRs did not report use of a SR protocol, eligibility criteria relating to publication status, years of coverage of the search, a full Boolean search logic for at least one database, methods for data extraction, methods for study risk of bias assessment, a primary outcome, an abstract conclusion that incorporated study limitations, or the funding source of the SR. Cochrane SRs, which accounted for 15% of the sample, had more complete reporting than all other types of SRs. Reporting has generally improved since 2004, but remains suboptimal for many characteristics.ConclusionsAn increasing number of SRs are being published, and many are poorly conducted and reported. Strategies are needed to help reduce this avoidable waste in research.

Objective To explore whether systematic reviewers selectively include trial effect estimates in meta-analyses when multiple are available, and what impact this may have on meta-analytic effects.Design Cross-sectional study.Data sources We randomly selected systematic reviews of interventions from 2 clinical specialties published between January 2010 and 2012. The first presented meta-analysis of a continuous outcome in each review was selected (index meta-analysis), and all trial effect estimates that were eligible for inclusion in the meta-analysis (eg, from multiple scales or time points) were extracted from trial reports.Analysis We calculated a statistic (the Potential Bias Index (PBI)) to quantify and test for evidence of selective inclusion. The PBI ranges from 0 to 1; values above or below 0.5 are suggestive of selective inclusion of effect estimates more or less favourable to the intervention, respectively. The impact of any potential selective inclusion was investigated by comparing the index meta-analytic standardised mean difference (SMD) to the median of a randomly constructed distribution of meta-analytic SMDs (representing the meta-analytic SMD expected when there is no selective inclusion).Results 31 reviews (250 trials) were included. The estimated PBI was 0.57 (95% CI 0.50 to 0.63), suggesting that trial effect estimates that were more favourable to the intervention were included in meta-analyses slightly more often than expected under a process consistent with random selection; however, the 95% CI included the null hypothesis of no selective inclusion. Any potential selective inclusion did not have an important impact on the meta-analytic effects.Conclusion There was no clear evidence that selective inclusion of trial effect estimates occurred in this sample of meta-analyses. Further research on selective inclusion in other clinical specialties is needed. To enable readers to assess the risk of selective inclusion bias, we recommend that systematic reviewers report the methods used to select effect estimates to include in meta-analyses.

ObjectiveTo synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials.DesignSystematic review of meta-epidemiological studies.MethodsWe retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ("mortality" versus "other objective" versus "subjective"). Direction of effect was standardised so that ROR < 1 and dSMD < 0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic.ResultsWe included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear.ConclusionsCertain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed.

IntroductionThis cluster randomised trial evaluated an intervention to decrease x-ray referrals and increase giving advice to stay active for people with acute low back pain (LBP) in general practice.MethodsGeneral practices were randomised to either access to a guideline for acute LBP (control) or facilitated interactive workshops (intervention). We measured behavioural predictors (e.g. knowledge, attitudes and intentions) and fear avoidance beliefs. We were unable to recruit sufficient patients to measure our original primary outcomes so we introduced other outcomes measured at the general practitioner (GP) level: behavioural simulation (clinical decision about vignettes) and rates of x-ray and CT-scan (medical administrative data). All those not involved in the delivery of the intervention were blinded to allocation.Results47 practices (53 GPs) were randomised to the control and 45 practices (59 GPs) to the intervention. The number of GPs available for analysis at 12 months varied by outcome due to missing confounder information; a minimum of 38 GPs were available from the intervention group, and a minimum of 40 GPs from the control group. For the behavioural constructs, although effect estimates were small, the intervention group GPs had greater intention of practising consistent with the guideline for the clinical behaviour of x-ray referral. For behavioural simulation, intervention group GPs were more likely to adhere to guideline recommendations about x-ray (OR 1.76, 95%CI 1.01, 3.05) and more likely to give advice to stay active (OR 4.49, 95%CI 1.90 to 10.60). Imaging referral was not statistically significantly different between groups and the potential importance of effects was unclear; rate ratio 0.87 (95%CI 0.68, 1.10) for x-ray or CT-scan.ConclusionsThe intervention led to small changes in GP intention to practice in a manner that is consistent with an evidence-based guideline, but it did not result in statistically significant changes in actual behaviour.Trial registrationAustralian New Zealand Clinical Trials Registry ACTRN012606000098538.

Because certain groups at high risk for HIV/AIDS (human immunodeficiency virus/acquired immunodeficiency syndrome) come together in correctional facilities, seroprevalence was high early in the epidemic. The share of the HIV/AIDS epidemic borne by inmates of and persons released from jails and prisons in the United States (US) in 1997 was estimated in a previous paper. While the number of inmates and releasees has risen, their HIV seroprevalence rates have fallen. We sought to determine if the share of HIV/AIDS borne by inmates and releasees in the US decreased between 1997 and 2006. We created a new model of population flow in and out of correctional facilities to estimate the number of persons released in 1997 and 2006. In 1997, approximately one in five of all HIV-infected Americans was among the 7.3 million who left a correctional facility that year. Nine years later, only one in seven (14%) of infected Americans was among the 9.1 million leaving, a 29.3% decline in the share. For black and Hispanic males, two demographic groups with heightened incarceration rates, recently released inmates comprise roughly one in five of those groups' total HIV-infected persons, a figure similar to the proportion borne by the correctional population as a whole in 1997. Decreasing HIV seroprevalence among those admitted to jails and prisons, prolonged survival and aging of the US population with HIV/AIDS beyond the crime-prone years, and success with discharge planning programs targeting HIV-infected prisoners could explain the declining concentration of the epidemic among correctional populations. Meanwhile, the number of persons with HIV/AIDS leaving correctional facilities remains virtually identical. Jails and prisons continue to be potent targets for public health interventions. The fluid nature of incarcerated populations ensures that effective interventions will be felt not only in correctional facilities but also in communities to which releasees return.

Background Systematic reviews that aim to synthesize evidence on the effects of interventions targeted at populations often include interrupted time-series (ITS) studies. However, the suppression of ITS studies or results within these studies (known as reporting bias) has the potential to bias conclusions drawn in such systematic reviews, with potential consequences for healthcare decision-making. Therefore, we aim to determine whether there is evidence of reporting bias among ITS studies. Methods We will conduct a search for published protocols of ITS studies and reports of their results in PubMed, MEDLINE, and Embase up to December 31, 2022. We contact the authors of the ITS studies to seek information about their study, including submission status, data for unpublished results, and reasons for non-publication or non-reporting of certain outcomes. We will examine if there is evidence of publication bias by examining whether time-to-publication is influenced by the statistical significance of the study’s results for the primary research question using Cox proportional hazards regression. We will examine whether there is evidence of discrepancies in outcomes by comparing those specified in the protocols with those in the reports of results, and we will examine whether the statistical significance of an outcome’s result is associated with how completely that result is reported using multivariable logistic regression. Finally, we will examine discrepancies between protocols and reports of results in the methods by examining the data collection processes, model characteristics, and statistical analysis methods. Discrepancies will be summarized using descriptive statistics. Discussion These findings will inform systematic reviewers and policymakers about the extent of reporting biases and may inform the development of mechanisms to reduce such biases.

Abstract Background There is some evidence that corticosteroids may have a beneficial effect in hand osteoarthritis. We examined the efficacy of corticosteroids on symptoms and structural outcomes in hand osteoarthritis. Methods Ovid MEDLINE, Embase and Cochrane Central Register of Controlled Trials were searched from inception to October 2021 for randomized controlled trials investigating the efficacy of corticosteroids in hand osteoarthritis. Two authors independently screened records, extracted data, and assessed risk of bias using the RoB 2 tool. Standardized mean difference (SMD) or mean difference (MD) was calculated, and random-effects meta-analyses were performed. Results Of 13 included trials, 3 examined oral corticosteroids and clinical outcomes in any hand joints, 9 examined intra-articular injection of corticosteroids and clinical outcomes at the first carpometacarpal joint and one in the interphalangeal joints. In meta-analysis, oral corticosteroids reduced pain (SMD -0.53, 95% CI -0.79 to -0.28) and improved stiffness (MD -5.03, 95% CI -9.91 to -0.15; Australian Canadian Osteoarthritis Hand Index stiffness subscale) and function (SMD -0.37, 95% CI -0.63 to -0.12) at 4-6 weeks. However, there was no significant persistent effect on pain and function at 3 months which was 6-8 weeks after study medication was stopped. There was no significant effect of intra-articular corticosteroids on pain or function at 4-6 weeks or over 3-12 months in first carpometacarpal osteoarthritis. Two trials evaluated joint structure at 4-6 weeks: one study showed oral corticosteroids reduced synovial thickening, neither showed an effect on synovitis. Conclusions There was low-certainty evidence for a medium effect of oral corticosteroids on pain relief and stiffness improvement and small-to-medium effect on functional improvement at 4-6 weeks, with no significant effect for intra-articular corticosteroids. Corticosteroids had no significant effect on any outcomes over longer term (3-12 months) off treatment. No trials examined the effect of corticosteroids on disease progression. The role of corticosteroids in hand osteoarthritis is limited.

Background: While the PRISMA 2020 statement is intended to guide the reporting of original systematic reviews, updated systematic reviews, and living systematic reviews (LSRs), its explanation and elaboration document notes that additional considerations for updated systematic reviews and LSRs may need to be addressed. This paper reports the protocol for developing an extension of the PRISMA 2020 statement for LSRs. Methods: We will follow the EQUATOR Network’s guidance for developing health research reporting guidelines. We will review the literature to identify possible items of the PRISMA 2020 checklist that need modification, as well as new items that need to be added. Then, we will survey representatives of different stakeholder groups for their views on the proposed modifications of the PRISMA 2020 checklist. We will summarize, present, and discuss the results of the survey in an online meeting, aiming to reach consensus on the content of the LSR extension. We will then draft the checklist, explanation and elaboration for each item, and flow diagram for the PRISMA 2020 extension. Then, we will share these initial documents with stakeholder representatives for final feedback and approval. Discussion: We anticipate that the PRISMA 2020 extension for LSRs will benefit LSR authors, editors, and peer reviewers of LSRs, as well as different users of LSRs, including guideline developers, policy makers, healthcare providers, patients, and other stakeholders.

Background: While the PRISMA 2020 statement is intended to guide the reporting of original systematic reviews, updated systematic reviews, and living systematic reviews (LSRs), its explanation and elaboration document notes that additional considerations for updated systematic reviews and LSRs may need to be addressed. This paper reports the protocol for developing an extension of the PRISMA 2020 statement for LSRs. Methods: We will follow the EQUATOR Network’s guidance for developing health research reporting guidelines. We will review the literature to identify possible items of the PRISMA 2020 checklist that need modification, as well as new items that need to be added. Then, we will survey representatives of different stakeholder groups for their views on the proposed modifications of the PRISMA 2020 checklist. We will summarize, present, and discuss the results of the survey in an online meeting, aiming to reach consensus on the content of the LSR extension. We will then draft the checklist, explanation and elaboration for each item, and flow diagram for the PRISMA 2020 extension. Then, we will share these initial documents with stakeholder representatives for final feedback and approval. Discussion: We anticipate that the PRISMA 2020 extension for LSRs will benefit LSR authors, editors, and peer reviewers of LSRs, as well as different users of LSRs, including guideline developers, policy makers, healthcare providers, patients, and other stakeholders.

Numerous studies have demonstrated low but increasing rates of data and code sharing within medical and health research disciplines. However, it remains unclear how commonly data and code are shared across all fields of medical and health research, as well as whether sharing rates are positively associated with implementation of progressive policies by publishers and funders, or growing expectations from the medical and health research community at large. Therefore this systematic review aims to synthesise the findings of medical and health science studies that have empirically investigated the prevalence of data or code sharing, or both. Objectives include the investigation of: (i) the prevalence of public sharing of research data and code alongside published articles (including preprints), (ii) the prevalence of private sharing of research data and code in response to reasonable requests, and (iii) factors associated with the sharing of either research output (e.g., the year published, the publisher’s policy on sharing, the presence of a data or code availability statement). It is hoped that the results will provide some insight into how often research data and code are shared publicly and privately, how this has changed over time, and how effective some measures such as the institution of data sharing policies and data availability statements have been in motivating researchers to share their underlying data and code.

Numerous studies have demonstrated low but increasing rates of data and code sharing within medical and health research disciplines. However it remains unclear how commonly data and code are shared across all fields of medical and health research, as well as whether sharing rates are positively associated with implementation of progressive policies by publishers and funders, or growing expectations from the medical and health research community at large. Therefore this systematic review aims to synthesise the findings of medical and health science studies that have empirically investigated the prevalence of data or code sharing, or both. Objectives include the investigation of: (i) the prevalence of public sharing of research data and code alongside published articles (including preprints), (ii) the prevalence of private sharing of research data and code in response to reasonable requests, and (iii) factors associated with the sharing of either research output (e.g., the year published, the publisher’s policy on sharing, the presence of a data or code availability statement). It is hoped that the results will provide some insight into how often research data and code are shared publicly and privately, how this has changed over time, and how effective some measures such as the institution of data sharing policies and data availability statements have been in motivating researchers to share their underlying data and code.

Background: Funded health research is being published in journals that many regard as “predatory”, deceptive, and non-credible. We do not currently know whether funders provide guidance on how to select a journal in which to publish funded health research. Methods: We identified the largest 46 philanthropic, public, development assistance, public-private partnership, and multilateral funders of health research by expenditure, globally as well as four public funders from lower-middle income countries, from the list at https://healthresearchfunders.org. One of us identified guidance on disseminating funded research from each funders’ website (August/September 2017), then extracted information about selecting journals, which was verified by another assessor. Discrepancies were resolved by discussion. Results were summarized descriptively. This research used publicly available information; we did not seek verification with funding bodies. Results: The majority (44/50) of sampled funders indicated funding health research. 38 (of 44, 86%) had publicly available information about disseminating funded research, typically called “policies” (29, 76%). Of these 38, 36 (95%) mentioned journal publication for dissemination of which 13 (36.11%) offer variable guidance on selecting a journal, all of which relate to the funder’s open access mandate. Six funders (17%) outlined publisher requirements or features by which to select a journal. One funder linked to a document providing features of journals to look for (e.g. listed in the Directory of Open Access Journals) and to be wary of (e.g., no journal scope statement, uses direct and unsolicited marketing). Conclusions: Few funders provided guidance on how to select a journal in which to publish funded research. Funders have a duty to ensure that the research they fund is discoverable by others. This research is a benchmark for funder guidance on journal selection prior to the January 2021 implementation of Plan S (a global, funder-led initiative to ensure immediate, open access to funded, published research).

Background: Funded health research is being published in journals that many regard as “predatory”, deceptive, and non-credible. We do not currently know whether funders provide guidance on how to select a journal in which to publish funded health research. Methods: We identified the largest 46 philanthropic, public, development assistance, public-private partnership, and multilateral funders of health research by expenditure, globally as well as four public funders from lower-middle income countries, from the list at https://healthresearchfunders.org. One of us identified guidance on disseminating funded research from each funders’ website (August/September 2017), then extracted information about selecting journals, which was verified by another assessor. Discrepancies were resolved by discussion. Results were summarized descriptively. This research used publicly available information; we did not seek verification with funding bodies. Results: The majority (44/50) of sampled funders indicated funding health research. 38 (of 44, 86%) had publicly available information about disseminating funded research, typically called “policies” (29, 76%). Of these 38, 36 (95%) mentioned journal publication for dissemination of which 13 (36.11%) offer variable guidance on selecting a journal, all of which relate to the funder’s open access mandate. Six funders (17%) outlined publisher requirements or features by which to select a journal. One funder linked to a document providing features of journals to look for (e.g. listed in the Directory of Open Access Journals) and to be wary of (e.g., no journal scope statement, uses direct and unsolicited marketing). Conclusions: Few funders provided guidance on how to select a journal in which to publish funded research. Funders have a duty to ensure that the research they fund is discoverable by others. This research is a benchmark for funder guidance on journal selection prior to the January 2021 implementation of Plan S (a global, funder-led initiative to ensure immediate, open access to funded, published research).

Background: Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. Objectives: To explore in SRs with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available; (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects, and; (iii) the risk of bias due to missing results (publication bias and selective non-reporting bias) in meta-analyses. Methods: We will systematically search for SRs with meta-analysis of the association between food/diet and health-related outcomes in a generally healthy population, published between January 2018 and June 2019. We will randomly sort titles and abstracts and screen them until we identify 50 eligible SRs. The first reported meta-analysis of a binary or continuous outcome in each SR (the ‘index meta-analysis’) will be evaluated. We will extract from study reports all study effect estimates that were eligible for inclusion in the index meta-analyses (e.g. from multiple instruments and time points) and will quantify and test for evidence of selective inclusion of results. We will also assess the risk of bias due to missing results in the index meta-analyses using a new tool (ROB-ME). Ethics and dissemination: Ethics approval is not required because information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. We will make all data collected from this study publicly available via the Open Science Framework.

Background: Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. Objectives: To explore in SRs with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available; (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects, and; (iii) the risk of bias due to missing results (publication bias and selective non-reporting bias) in meta-analyses. Methods: We will systematically search for SRs with meta-analysis of the association between food/diet and health-related outcomes in a generally healthy population, published between January 2018 and June 2019. We will randomly sort titles and abstracts and screen them until we identify 50 eligible SRs. The first reported meta-analysis of a binary or continuous outcome in each SR (the ‘index meta-analysis’) will be evaluated. We will extract from study reports all study effect estimates that were eligible for inclusion in the index meta-analyses (e.g. from multiple instruments and time points) and will quantify and test for evidence of selective inclusion of results. We will also assess the risk of bias due to missing results in the index meta-analyses using a new tool (ROB-ME). Ethics and dissemination: Ethics approval is not required because information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. We will make all data collected from this study publicly available via the Open Science Framework.