Search

ObjectiveTo assess the methodological and reporting quality of systematic reviews (SRs) that informed recommendations in the recent American and European hypertension guidelines.Design and settingsMeta-epidemiological study. We identified SRs that were cited for class I recommendations based on Level of Evidence-A in the 2017 American College of Cardiology/American Heart Association (ACC/AHA) and the 2018 European Society of Cardiology/European Society of Hypertension (ESC/ESH) hypertension guidelines.Main outcome measuresMethodological and reporting quality of the SRs was assessed using A Measurement Tool to Assess Systematic Reviews (AMSTAR-2) checklist and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist, respectively.ResultsA total of 40 SRs was included in the analysis (28 from 2017 ACC/AHA; 22 from 2018 ESC/ESH and 10 were included in both). Based on the AMSTAR-2 assessment, only 7.5% SRs were found to be of high methodological quality, 47.5% were of moderate, each 22.5% were of low and critically low quality. Based on the PRISMA checklist assessment, a mean of 24 items (SD (2.76) were reported appropriately, and only five SRs reported all 27 items appropriately.ConclusionMethodological and reporting quality of SRs were found to vary considerably. Lack of information on the funding source of included studies, use of a protocol, integration of risk of bias assessments while interpreting findings and reporting of excluded studies were major methodological deficiencies.

• Given the urgent need for credible answers to high-priority questions about the health and social impacts of COVID-19, many systematic reviewers seek to contribute their skills and expertise. • Rather than embarking on unnecessary, duplicate reviews, we encourage the evidence synthesis community to prioritise purposeful replication of systematic reviews of evidence relevant to COVID-19. • We explain why replication of systematic reviews is important, how to carry out a replication, and when to consider replication of reviews.

Background Evidence based medicine aims to integrate scientific evidence, clinical experience, and patient values and preferences. Individual health care professionals need to appraise the evidence from randomized trials and observational studies when guidelines are not yet available. To date, tools for assessment of bias and terminologies for bias are specific for each study design. Moreover, most tools appeal only to methodological knowledge to detect bias, not to subject matter knowledge, i.e. in-depth medical knowledge about a topic. We propose a unified framework that enables the coherent assessment of bias across designs. Methods Epidemiologists traditionally distinguish between three types of bias in observational studies: confounding, information bias, and selection bias. These biases result from a common cause, systematic error in the measurement or common effect of the intervention and outcome respectively. We applied this conceptual framework to randomized trials and show how it can be used to identify bias. The three sources of bias were illustrated with graphs that visually represent researchers’ assumptions about the relationships between the investigated variables (causal diagrams). Results Critical appraisal of evidence started with the definition of the research question in terms of the population of interest, the compared interventions and the main outcome. Next, we used causal diagrams to illustrate how each source of bias can lead to over- or underestimated treatment effects. Then, we discussed how randomization, blinded outcome measurement and intention-to-treat analysis minimize bias in trials. Finally, we identified study aspects that can only be appraised with subject matter knowledge, irrespective of study design. Conclusions The unified framework encompassed the three main sources of bias for the effect of an assigned intervention on an outcome. It facilitated the integration of methodological and subject matter knowledge in the assessment of bias. We hope that graphical diagrams will help clarify debate among professionals by reducing misunderstandings based on different terminology for bias.

Objective The objective of this study was to identify potential trial characteristics associated with reported treatment effect estimates in randomized trials of testosterone therapy in adult men. Study Design and Setting This is a meta-epidemiological study. MEDLINE was searched for meta-analyses of randomized trials of testosterone therapy in men published between 2008 and 2018. Data on trial characteristics were extracted independently by two reviewers. The impact of trial characteristics on reported treatment effects was investigated using a two-step meta-analytic approach. Results We identified 132 randomized trials, included in 19 meta-analyses, comprising data from 10,725 participants. None of the investigated design characteristics, including year of publication, sample size, trial registration status, center status, regionality, funding source, and conflict of interest were statistically significantly associated with reported treatment effects of testosterone therapy in men. Although trials rated at high risk of bias overall reported treatment effects that were 21% larger compared with trials rated at low risk of bias overall, the 95% confidence interval included the null (ratio of odds ratio: 0.79, 95% confidence interval: 0.60 to 1.03). Conclusion The present study found no clear evidence that trial characteristics are associated with treatment effects in randomized trials of testosterone therapy in men. To establish stronger evidence about the treatment effects of testosterone therapy in men, future randomized trials should not only be adequately designed but also transparently reported. Study Registration osf.io/x9g6m.

Synthesizing evidence is an essential part of scientific progress, but it is often done in a slow and uncoordinated manner, sometimes producing misleading conclusions. Here, we propose the idea of an ‘open synthesis community’ to resolve this pressing issue.

Objectives The aim of the study was to generate a comprehensive bank of systematic review (SR) reporting items to inform an update of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2009 statement. Methods We searched the Enhancing the QUAlity and Transparency Of health Research Network library in May 2019 to identify all reporting guidelines for SRs that were published after 2009, regardless of the scope of the guideline. We also conducted a selective review of four guidance manuals for SRs, three tools for assessing the risk of bias in SRs, six meta-research studies evaluating the reporting quality of SRs using a tailored checklist, and five reporting guidelines for other study designs. One author screened and selected sources for inclusion, extracted reporting guidance from sources, and mapped guidance against the PRISMA 2009 checklist items. Results We included 60 sources providing guidance on reporting of SRs and meta-analyses. From these, we collated a list of 221 unique reporting items. Items were categorized into title (four items), abstract (10 items), introduction (12 items), methods (111 items), results (61 items), discussion (12 items), funding and conflicts of interest (four items), administrative information (three items), and data availability (four items). This exercise generated 175 reporting items that could be added to the guidance in the PRISMA 2009 statement. Conclusion Generation of a comprehensive item bank through review and mapping of the literature facilitates identification of missing items and those needing modification, which may not otherwise be identified by the guideline development team or from other activities commonly used to develop reporting guidelines.

Background Standard practice for conducting systematic reviews (SRs) is time consuming and involves the study team screening hundreds or thousands of citations. As the volume of medical literature grows, the citation set sizes and corresponding screening efforts increase. While larger team size and alternate screening methods have the potential to reduce workload and decrease SR completion times, it is unknown whether investigators adapt team size or methods in response to citation set sizes. Using a cross-sectional design, we sought to understand how citation set size impacts (1) the total number of authors or individuals contributing to screening and (2) screening methods. Methods MEDLINE was searched in April 2019 for SRs on any health topic. A total of 1880 unique publications were identified and sorted into five citation set size categories (after deduplication): 10,000. A random sample of 259 SRs were selected (~ 50 per category) for data extraction and analysis. Results With the exception of the pairwise t test comparing the under 1000 and over 10,000 categories (median 5 vs. 6, p = 0.049) no statistically significant relationship was evident between author number and citation set size. While visual inspection was suggestive, statistical testing did not consistently identify a relationship between citation set size and number of screeners (title-abstract, full text) or data extractors. However, logistic regression identified investigators were significantly more likely to deviate from gold-standard screening methods (i.e. independent duplicate screening) with larger citation sets. For every doubling of citation size, the odds of using gold-standard screening decreased by 15 and 20% at title-abstract and full text review, respectively. Finally, few SRs reported using crowdsourcing (n = 2) or computer-assisted screening (n = 1). Conclusions Large citation set sizes present a challenge to SR teams, especially when faced with time-sensitive health policy questions. Our study suggests that with increasing citation set size, authors are less likely to adhere to gold-standard screening methods. It is possible that adjunct screening methods, such as crowdsourcing (large team) and computer-assisted technologies, may provide a viable solution for authors to complete their SRs in a timely manner.

ObjectiveTo compare results reporting and the presence of spin in COVID-19 study preprints with their finalised journal publications.DesignCross-sectional study.SettingInternational medical literature.ParticipantsPreprints and final journal publications of 67 interventional and observational studies of COVID-19 treatment or prevention from the Cochrane COVID-19 Study Register published between 1 March 2020 and 30 October 2020.Main outcome measuresStudy characteristics and discrepancies in (1) results reporting (number of outcomes, outcome descriptor, measure, metric, assessment time point, data reported, reported statistical significance of result, type of statistical analysis, subgroup analyses (if any), whether outcome was identified as primary or secondary) and (2) spin (reporting practices that distort the interpretation of results so they are viewed more favourably).ResultsOf 67 included studies, 23 (34%) had no discrepancies in results reporting between preprints and journal publications. Fifteen (22%) studies had at least one outcome that was included in the journal publication, but not the preprint; eight (12%) had at least one outcome that was reported in the preprint only. For outcomes that were reported in both preprints and journals, common discrepancies were differences in numerical values and statistical significance, additional statistical tests and subgroup analyses and longer follow-up times for outcome assessment in journal publications.At least one instance of spin occurred in both preprints and journals in 23/67 (34%) studies, the preprint only in 5 (7%), and the journal publications only in 2 (3%). Spin was removed between the preprint and journal publication in 5/67 (7%) studies; but added in 1/67 (1%) study.ConclusionsThe COVID-19 preprints and their subsequent journal publications were largely similar in reporting of study characteristics, outcomes and spin. All COVID-19 studies published as preprints and journal publications should be critically evaluated for discrepancies and spin.

Background: There are numerous guidelines developed for bone health. Yet, it is unclear whether the differences in guideline development methods explain the variability in the recommendations for vitamin D and calcium intake. The objective of this systematic review was to collate and compare recommendations for vitamin D and calcium across bone health guidelines, assess the methods used to form the recommendations, and explore which methodological factors were associated with these guideline recommendations. Methods: We searched MEDLINE, EMBASE, CINAHL, and other databases indexing guidelines to identify records in English between 2009 and 2019. Guidelines or policy statements on bone health or osteoporosis prevention for generally healthy adults aged ≥40 years were eligible for inclusion. Two reviewers independently extracted recommendations on daily vitamin D and calcium intake, supplement use, serum 25 hydroxyvitamin D [25(OH)D] level, and sunlight exposure; assessed guideline development methods against 25 recommended criteria in the World Health Organization (WHO) handbook for guideline development; and, identified types identified types of evidence underpinning the recommendations. Results: we included 47 eligible guidelines from 733 records: 74% of the guidelines provided vitamin D (200~600–4000 IU/day) and 70% provided calcium (600–1200 mg/day) recommendations, 96% and 88% recommended vitamin D and calcium supplements, respectively, and 70% recommended a specific 25(OH)D concentration. On average, each guideline met 10 (95% CI: 9–12) of the total of 25 methodological criteria for guideline development recommended by the WHO Handbook. There was uncertainty in the association between the methodological criteria and the proportion of guidelines that provided recommendations on daily vitamin D or calcium. Various types of evidence, including previous bone guidelines, nutrient reference reports, systematic reviews, observational studies, and perspectives/editorials were used to underpin the recommendations. Conclusions: There is considerable variability in vitamin D and calcium recommendations and in guideline development methods in bone health guidelines. Effort is required to strengthen the methodological rigor of guideline development and utilize the best available evidence to underpin nutrition recommendations in evidence-based guidelines on bone health.

Background: Systematic reviews underpin clinical practice and policies that guide healthcare decisions. A core component of many systematic reviews is meta-analysis, which is a statistical synthesis of results across studies. Errors in the conduct and interpretation of meta-analysis can lead to incorrect conclusions regarding the benefits and harms of interventions; and studies have shown that these errors are common. Enabling peer reviewers to better detect errors in meta-analysis through the use of a checklist provides an opportunity for these errors to be rectified before publication. To our knowledge, no such checklist exists. Objective: To develop and evaluate a checklist to detect errors in pairwise meta-analyses in systematic reviews of interventions. Methods: We will undertake a four-step process to develop the checklist. First, we will undertake a systematic review of studies that have evaluated errors in the conduct and interpretation of meta-analysis to generate a bank of items to consider for the checklist. Second, we will undertake a survey of systematic review methodologists and statisticians to seek their views on which items, of the bank of items generated in step 1, are most important to include in the checklist. Third, we will hold a virtual meeting to agree upon which items to include in the checklist. Fourth, before finalising the checklist, we will pilot with editors and peer reviewers of journals. Conclusion: The developed checklist is intended to help journal editors and peer reviewers identify errors in the application and interpretation of meta-analyses in systematic reviews. Fewer errors in the conduct and improved interpretation will lead to more accurate review findings and conclusions to inform clinical practice.

Objective.To describe the experience of the first OMERACT Emerging Leaders Program (ELP).Methods.A Delphi process identified positive aspects, areas for improvement, and future directions. Core items were defined as essential if they received ≥ 70% ratings.Results.Participants valued relatable/accessible mentors (100%), including an OMERACT Executive mentor (100%), and a support network of peers (90%). Key items for future development were funding support (100%) and developing knowledge about OMERACT processes (90%) and politics (80%).Conclusion.The ELP has the potential to provide targeted training for early career researchers to develop relevant skills for future leadership roles within OMERACT.

Objectives The aim of the article was to identify and summarize studies assessing methodologies for study selection, data abstraction, or quality appraisal in systematic reviews. Study Design and Setting A systematic review was conducted, searching MEDLINE, EMBASE, and the Cochrane Library from inception to September 1, 2016. Quality appraisal of included studies was undertaken using a modified Quality Assessment of Diagnostic Accuracy Studies 2, and key results on accuracy, reliability, efficiency of a methodology, or impact on results and conclusions were extracted. Results After screening 5,600 titles and abstracts and 245 full-text articles, 37 studies were included. For screening, studies supported the involvement of two independent experienced reviewers and the use of Google Translate when screening non-English articles. For data abstraction, studies supported involvement of experienced reviewers (especially for continuous outcomes) and two independent reviewers, use of dual monitors, graphical data extraction software, and contacting authors. For quality appraisal, studies supported intensive training, piloting quality assessment tools, providing decision rules for poorly reported studies, contacting authors, and using structured tools if different study designs are included. Conclusion Few studies exist documenting common systematic review practices. Included studies support several systematic review practices. These results provide an updated evidence-base for current knowledge synthesis guidelines and methods requiring further research.

Objectives To illustrate the use of process mining concepts, techniques, and tools to improve the systematic review process. Study Design and Setting We simulated review activities and step-specific methods in the process for systematic reviews conducted by one research team over 1 year to generate an event log of activities, with start/end dates, reviewer assignment by expertise, and person-hours worked. Process mining techniques were applied to the event log to “discover” process models, which allowed visual display, animation, or replay of the simulated review activities. Summary statistics were calculated for person-time and timelines. We also analyzed the social networks of team interactions. Results The 12 simulated reviews included an average of 3,831 titles/abstracts (range: 1,565–6,368) and 20 studies (6–42). The average review completion time was 463 days (range: 289–629) (881 person-hours [range: 243–1,752]). The average person-hours per activity were study selection 26%, data collection 24%, report preparation 23%, and meta-analysis 17%. Social network analyses showed the organizational interaction of team members, including how they worked together to complete review tasks and to hand over tasks upon completion. Conclusion Event log and process mining can be valuable tools for research teams interested in improving and modernizing the systematic review process.

Background: Funded health research is being published in journals that many regard as “predatory”, deceptive, and non-credible. We do not currently know whether funders provide guidance on how to select a journal in which to publish funded health research. Methods: We identified the largest 46 philanthropic, public, development assistance, public-private partnership, and multilateral funders of health research by expenditure, globally as well as four public funders from lower-middle income countries, from the list at https://healthresearchfunders.org. One of us identified guidance on disseminating funded research from each funders’ website (August/September 2017), then extracted information about selecting journals, which was verified by another assessor. Discrepancies were resolved by discussion. Results were summarized descriptively. This research used publicly available information; we did not seek verification with funding bodies. Results: The majority (44/50) of sampled funders indicated funding health research. 38 (of 44, 86%) had publicly available information about disseminating funded research, typically called “policies” (29, 76%). Of these 38, 36 (95%) mentioned journal publication for dissemination of which 13 (36.11%) offer variable guidance on selecting a journal, all of which relate to the funder’s open access mandate. Six funders (17%) outlined publisher requirements or features by which to select a journal. One funder linked to a document providing features of journals to look for (e.g. listed in the Directory of Open Access Journals) and to be wary of (e.g., no journal scope statement, uses direct and unsolicited marketing). Conclusions: Few funders provided guidance on how to select a journal in which to publish funded research. Funders have a duty to ensure that the research they fund is discoverable by others. This research is a benchmark for funder guidance on journal selection prior to the January 2021 implementation of Plan S (a global, funder-led initiative to ensure immediate, open access to funded, published research).

Background: The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement, published in 2009, was designed to help systematic reviewers transparently report why the review was done, what the authors did and what they found. Over the last decade, there have been many advances in systematic review methodology and terminology, which have necessitated an update to the guideline.Objectives: To develop the PRISMA 2020 statement for reporting systematic reviews.Methods: We reviewed 60 documents with reporting guidance for systematic reviews to generate suggested modifications to the PRISMA 2009 statement. We sought feedback on the suggested modifications through an online survey of 110 systematic review methodologists and journal editors. The results of the review and survey were discussed at a 21-member in-person meeting. Following the meeting, drafts of the PRISMA 2020 checklist, abstract checklist, explanation and elaboration and flow diagram were generated and refined iteratively based on feedback from co-authors and a convenience sample of 15 systematic reviewers.Results: In this statement paper, we present the PRISMA 2020 27-item checklist, an expanded checklist that details reporting recommendations for each item, the PRISMA 2020 abstract checklist, and the revised flow diagrams for original and updated reviews. The checklist includes new reporting guidance that reflects advances in methods to identify, select, appraise and synthesise studies. The structure and presentation of the items have been modified to facilitate implementation. The PRISMA 2020 statement replaces the 2009 statement.Conclusions: The PRISMA 2020 statement is intended to facilitate transparent, complete and accurate reporting of systematic reviews. Improved reporting should benefit users of reviews, including guideline developers, policy makers, health care providers, patients and other stakeholders. In order to achieve this, we encourage authors, editors and peer-reviewers to adopt the guideline.

The methods and results of systematic reviews should be reported in sufficient detail to allow users to assess the trustworthiness and applicability of the review findings. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement was developed to facilitate transparent and complete reporting of systematic reviews and has been updated (to PRISMA 2020) to reflect recent advances in systematic review methodology and terminology. Here, we present the explanation and elaboration paper for PRISMA 2020, where we explain why reporting of each item is recommended, present bullet points that detail the reporting recommendations, and present examples from published reviews. We hope that changes to the content and structure of PRISMA 2020 will facilitate uptake of the guideline and lead to more transparent, complete, and accurate reporting of systematic reviews.

Background Investigations of transparency, reproducibility and replicability in science have been directed largely at individual studies. It is just as critical to explore these issues in syntheses of studies, such as systematic reviews, given their influence on decision-making and future research. We aim to explore various aspects relating to the transparency, reproducibility and replicability of several components of systematic reviews with meta-analysis of the effects of health, social, behavioural and educational interventions. Methods The REPRISE (REProducibility and Replicability In Syntheses of Evidence) project consists of four studies. We will evaluate the completeness of reporting and sharing of review data, analytic code and other materials in a random sample of 300 systematic reviews of interventions published in 2020 (Study 1). We will survey authors of systematic reviews to explore their views on sharing review data, analytic code and other materials and their understanding of and opinions about replication of systematic reviews (Study 2). We will then evaluate the extent of variation in results when we (a) independently reproduce meta-analyses using the same computational steps and analytic code (if available) as used in the original review (Study 3), and (b) crowdsource teams of systematic reviewers to independently replicate a subset of methods (searches for studies, selection of studies for inclusion, collection of outcome data, and synthesis of results) in a sample of the original reviews; 30 reviews will be replicated by 1 team each and 2 reviews will be replicated by 15 teams (Study 4). Discussion The REPRISE project takes a systematic approach to determine how reliable systematic reviews of interventions are. We anticipate that results of the REPRISE project will inform strategies to improve the conduct and reporting of future systematic reviews.

In an effort to better utilize published evidence obtained from animal experiments, systematic reviews of preclinical studies are increasingly more common—along with the methods and tools to appraise them (e.g., SYstematic Review Center for Laboratory animal Experimentation [SYRCLE’s] risk of bias tool). We performed a cross-sectional study of a sample of recent preclinical systematic reviews (2015–2018) and examined a range of epidemiological characteristics and used a 46-item checklist to assess reporting details. We identified 442 reviews published across 43 countries in 23 different disease domains that used 26 animal species. Reporting of key details to ensure transparency and reproducibility was inconsistent across reviews and within article sections. Items were most completely reported in the title, introduction, and results sections of the reviews, while least reported in the methods and discussion sections. Less than half of reviews reported that a risk of bias assessment for internal and external validity was undertaken, and none reported methods for evaluating construct validity. Our results demonstrate that a considerable number of preclinical systematic reviews investigating diverse topics have been conducted; however, their quality of reporting is inconsistent. Our study provides the justification and evidence to inform the development of guidelines for conducting and reporting preclinical systematic reviews., A cross sectional study of a sample of recent preclinical systematic reviews reveals deficiencies in reporting and provides the justification and evidence to inform the development of specific guidelines for conducting and reporting preclinical systematic reviews.

For systematic reviews of interventions, replication is defined as the reproduction of findings of previous systematic reviews looking at the same effectiveness question either by: purposefully repeating the same methods to verify one or more empirical findings; or purposefully extending or narrowing the systematic review to a broader or more focused question (eg, across broader or more focused populations, intervention types, settings, outcomes, or study designs) Although systematic reviews are often used as the basis for informing policy and practice decisions, little evidence has been published so far on whether replication of systematic reviews is worthwhile Replication of existing systematic reviews cannot be done for all topics; any unnecessary or poorly conducted replication contributes to research waste The decision to replicate a systematic review should be based on the priority of the research question; the likelihood that a replication will resolve uncertainties, controversies, or the need for additional evidence; the magnitude of the benefit or harm of implementing findings of a replication; and the opportunity cost of the replication Systematic review authors, commissioners, funders, and other users (including clinicians, patients, and representatives from policy making organisations) can use the guidance and checklist proposed here to assess the need for a replication

Objectives To describe the processes used to update the PRISMA 2009 statement for reporting systematic reviews, present results of a survey conducted to inform the update, summarize decisions made at the PRISMA update meeting, and describe and justify changes made to the guideline. Methods We reviewed 60 documents with reporting guidance for systematic reviews to generate suggested modifications to the PRISMA 2009 statement. We invited 220 systematic review methodologists and journal editors to complete a survey about the suggested modifications. The results of these projects were discussed at a 21-member in-person meeting. Following the meeting, we drafted the PRISMA 2020 statement and refined it based on feedback from co-authors and a convenience sample of 15 systematic reviewers. Results The review of 60 documents revealed that all topics addressed by the PRISMA 2009 statement could be modified. Of the 110 survey respondents, more than 66% recommended keeping six of the original checklist items as they were and modifying 15 of them using wording suggested by us. Attendees at the in-person meeting supported the revised wording for several items but suggested rewording for most to enhance clarity, and further refinements were made over six drafts of the guideline. Conclusions The PRISMA 2020 statement consists of updated reporting guidance for systematic reviews. We hope that providing this detailed description of the development process will enhance the acceptance and uptake of the guideline and assist those developing and updating future reporting guidelines.

• Open Science principles are vital for ensuring reproducibility, trust, and legacy. • Evidence synthesis is a vital means of summarizing research for decision-making. • Open Synthesis is the application of Open Science principles to evidence synthesis. • Open approaches to planning, conducting, and reporting synthesis have many benefits. • We call on the evidence synthesis community to embrace Open Synthesis.

Background Understanding the long-term health effects of low to moderate alcohol consumption is important for establishing thresholds for minimising the lifetime risk of harm. Recent research has elucidated the dose-response relationship between alcohol and cardiovascular outcomes, showing an increased risk of harm at levels of intake previously thought to be protective. The primary objective of this review was to examine (1) whether there is a dose-response relationship between levels of alcohol consumption and long-term cognitive effects, and (2) what the effects are of different levels of consumption. Methods The review was conducted according to a pre-specified protocol. Eligible studies were those published 2007 onwards that compared cognitive function among people with different levels of alcohol consumption (measured ≥ 6 months prior to first follow-up of cognition). Major cognitive impairment was excluded. Searches were limited to MEDLINE, Embase and PsycINFO (January 2007 to April 2018). Screening, data extraction, and risk of bias assessment (ROBINS-I) were piloted by three authors, then completed by a single author and checked by a second. Analyses were undertaken to identify and characterise dose-response relationships between levels of alcohol consumption and cognition. Certainty of evidence was assessed using GRADE. Results We included 27 cohort studies (from 4786 citations). Eighteen studies examined the effects of alcohol consumption at different levels (risk of bias 16 serious, 2 critical). Ten studies provided data for dose-response analysis. The pooled dose-response relationship showed a maximum standardised mean difference (SMD) indicating slightly better cognition among women with moderate alcohol consumption compared to current non-drinkers (SMD 0.18, 95%CI 0.02 to 0.34, at 14.4 grams/day; 5 studies, very low certainty evidence), and a trivial difference for men (SMD 0.05, 95% CI 0.00 to 0.10, at 19.4 grams/day; 6 studies, very low certainty evidence). Conclusions Major limitations in the design and reporting of included studies made it impossible to discern if the effects of ‘lower’ levels of alcohol intake are due to bias. Further review of the evidence is unlikely to resolve this issue without meta-analysis of individual patient data from cohort studies that address biases in the selection of participants and classification of alcohol consumption.

Objectives To evaluate how often reproducible research practices, which allow others to recreate the findings of studies, given the original data, are used in systematic reviews (SRs) of biomedical research. Study Design and Setting We evaluated a random sample of SRs indexed in MEDLINE during February 2014, which focused on a therapeutic intervention and reported at least one meta-analysis. Data on reproducible research practices in each SR were extracted using a 26-item form by one author, with a 20% random sample extracted in duplicate. We explored whether the use of reproducible research practices was associated with an SR being a Cochrane review, as well as with the reported use of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. Results We evaluated 110 SRs of therapeutic interventions, 78 (71%) of which were non-Cochrane SRs. Across the SRs, there were 2,139 meta-analytic effects (including subgroup meta-analytic effects and sensitivity analyses), 1,551 (73%) of which were reported in sufficient detail to recreate them. Systematic reviewers reported the data needed to recreate all meta-analytic effects in 72 (65%) SRs only. This percentage was higher in Cochrane than in non-Cochrane SRs (30/32 [94%] vs. 42/78 [54%]; risk ratio 1.74, 95% confidence interval 1.39–2.18). Systematic reviewers who reported imputing, algebraically manipulating, or obtaining some data from the study author/sponsor infrequently stated which specific data were handled in this way. Only 33 (30%) SRs mentioned access to data sets and statistical code used to perform analyses. Conclusion Reproducible research practices are underused in SRs of biomedical interventions. Adoption of such practices facilitates identification of errors and allows the SR data to be reanalyzed.

Numerous studies have demonstrated low but increasing rates of data and code sharing within medical and health research disciplines. However, it remains unclear how commonly data and code are shared across all fields of medical and health research, as well as whether sharing rates are positively associated with implementation of progressive policies by publishers and funders, or growing expectations from the medical and health research community at large. Therefore this systematic review aims to synthesise the findings of medical and health science studies that have empirically investigated the prevalence of data or code sharing, or both. Objectives include the investigation of: (i) the prevalence of public sharing of research data and code alongside published articles (including preprints), (ii) the prevalence of private sharing of research data and code in response to reasonable requests, and (iii) factors associated with the sharing of either research output (e.g., the year published, the publisher’s policy on sharing, the presence of a data or code availability statement). It is hoped that the results will provide some insight into how often research data and code are shared publicly and privately, how this has changed over time, and how effective some measures such as the institution of data sharing policies and data availability statements have been in motivating researchers to share their underlying data and code.

IntroductionComplete reporting of systematic reviews of interventions is essential to the interpretation of research findings and the reproducibility of research results. The Template for Intervention Description and Replication (TIDieR) checklist—and the version specific to systematic reviews (TIDieR-SR)—was created to provide authors and researchers an evidence-based guide for reporting trial and systematic review interventions. In this study, we apply TIDieR-SR to Cochrane systematic reviews of surgical interventions.MethodsWe searched the Cochrane Database for relevant systematic reviews. Two investigators applied inclusion/exclusion criteria to all titles/abstracts and full texts. These same investigators extracted all data in duplicate while masked to the other’s data. The primary outcome was adherence to TIDieR-SR items.ResultsTwo hundred and thirty-eight systematic reviews were included. Overall, included SRs adhered to a median of 6 (IQR 5–7) out of eight TIDieR-SR items. The item with the lowest adherence was item 7 (share intervention materials, 1/238 (0.4%).DiscussionOur results are encouraging, but the generalisability of our findings is compromised by the inclusion of only Cochrane systematic reviews. Future reporting of intervention materials is likely to improve the application of effective surgical interventions in the clinical practice.

OBJECTIVES To examine outcome reporting bias of systematic reviews registered in PROSPERO. STUDY DESIGN AND SETTING Retrospective cohort study. The primary outcomes from systematic review publications were compared with those reported in the corresponding PROSPERO records; discrepancies in the primary outcomes were assessed as upgrades, additions, omissions, or downgrades. Relative risks (RRs) and 95% confidence intervals (CI) were calculated to determine the likelihood of having a change in primary outcome when the meta-analysis result was favorable and statistically significant. RESULTS Ninety-six systematic reviews were published. A discrepancy in the primary outcome occurred in 32% of the included reviews and 39% of the reviews did not explicitly specify a primary outcome(s); 6% of the primary outcomes were omitted. There was no significant increased risk of adding/upgrading (RR, 2.14; 95% CI: 0.53, 8.63) or decreased risk of downgrading (RR, 0.76; 95% CI: 0.27, 2.17) an outcome when the meta-analysis result was favorable and statistically significant. As well, there was no significant increased risk of adding/upgrading (RR, 0.89; 95% CI: 0.31, 2.53) or decreased risk of downgrading (RR, 0.56; 95% CI: 0.29, 1.08) an outcome when the conclusion was positive. CONCLUSIONS We recommend review authors carefully consider primary outcome selection, and journals are encouraged to focus acceptance on registered systematic reviews.

Importance Reproducible research practices are essential to biomedical research because these practices promote trustworthy evidence. In systematic reviews and meta-analyses, reproducible research practices ensure that summary effects used to guide patient care are stable and trustworthy. Objective To evaluate the reproducibility in theory of meta-analyses in oncology systematic reviews cited by the 49 National Comprehensive Cancer Network (NCCN) guidelines for the treatment of cancer by site and evaluate whether Cochrane reviews or systematic reviews that report adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines use more reproducible research practices. Design, Setting, and Participants A cross-sectional investigation of all systematic reviews with at least 1 meta-analysis and at least 1 included randomized clinical trial (RCT) that are cited by NCCN guidelines for treatment of cancer by site. We scanned the reference list of all NCCN guidelines (n = 49) for potential systematic reviews and meta-analyses. All retrieved studies were screened, and data were extracted, independently and in duplicate. The analysis was carried out between May 6, 2018, and January 28, 2019. Main Outcomes and Measures The frequency of reproducible research practices, defined as (1) effect estimate and measure of precision (eg, hazard ratio with 95% confidence interval); (2) clear list of studies included for each analysis; and (3) for subgroup and sensitivity analyses, it must be clear which studies were included in each group or level. Results We identified 1124 potential systematic reviews, and 154 meta-analyses comprising 3696 meta-analytic effect size estimates were included. Only 2375 of the 3696 meta-analytic estimates (64.3%), including subgroup and sensitivity analyses, were reproducible in theory. Forest plots appear to improve the reproducibility of meta-analyses. All meta-analytic estimates were reproducible in theory in 100 systematic reviews (64.9%), and in 15 systematic reviews (9.7%), no meta-analytic estimates could potentially be reproduced. Data were said to be imputed in 29 meta-analyses, but none specified which data. Only 1 meta-analysis included a link to an online data set. Conclusions and Relevance More reproducible research practices are needed in oncology meta-analyses, as suggested by those that are cited by the NCCN. Reporting meta-analyses in forest plots and requirements for full data sharing are recommended.

Objectives To describe the experiences (including symptoms and perceived impacts on daily living) of people with a shoulder disorder. Methods Systematic review of qualitative studies. We searched for eligible qualitative studies indexed in Ovid MEDLINE, Ovid Embase, CINAHL (EBSCO), SportDiscus (EBSCO) and Ovid PsycINFO up until November 2017. Two authors independently screened studies for inclusion, appraised their methodological quality using the Critical Appraisal Skills Programme checklist, used thematic synthesis methods to generate themes describing the experiences reported by participants and assessed the confidence in the findings using the Grading of Recommendations Assessment, Development and Evaluation Confidence in Evidence from Reviews of Qualitative research (GRADE-CERQual) approach. Results The inclusion criteria were met by eight studies, which included 133 participants (49 females and 84 males) with either rotator cuff disease, adhesive capsulitis, proximal humeral fracture, shoulder instability or unspecified shoulder pain. We generated seven themes to describe what people in the included studies reported experiencing: pain; physical function/activity limitations; participation restriction; sleep disruption; cognitive dysfunction; emotional distress; and other pathophysiological manifestations (other than pain). There were interactions between the themes, with particular experiences impacting on others (e.g. pain leading to reduced activities and sleep disruption). Following grading of the evidence, we considered it likely that most of the review findings were a reasonable representation of the experiences of people with shoulder disorders. Conclusion Patients with shoulder disorders contend with considerable disruption to their life. The experiences described should be considered by researchers seeking to select the most appropriate outcomes to measure in clinical trials and other research studies in people with shoulder disorders.

Assessment of risk of bias is regarded as an essential component of a systematic review on the effects of an intervention. The most commonly used tool for randomised trials is the Cochrane risk-of-bias tool. We updated the tool to respond to developments in understanding how bias arises in randomised trials, and to address user feedback on and limitations of the original tool.

Objective.To reach consensus on the core domains to be included in a core domain set for clinical trials of shoulder disorders using the Outcome Measures in Rheumatology (OMERACT) Filter 2.1 Core Domain Set process.Methods.At OMERACT 2018, the OMERACT Shoulder Working Group conducted a workshop that presented the OMERACT 2016 preliminary core domain set and its rationale based upon a systematic review of domains measured in shoulder trials and international Delphi sessions involving patients, clinicians, and researchers, as well as a new systematic review of qualitative studies on the experiences of people with shoulder disorders. After discussions in breakout groups, the OMERACT core domain set for clinical trials of shoulder disorders was presented for endorsement by OMERACT 2018 participants.Results.The qualitative review (n = 8) identified all domains included in the preliminary core set. An additional domain, cognitive dysfunction, was also identified, but confidence that this represents a core domain was very low. The core domain set that was endorsed by the OMERACT participants, with 71% agreement, includes 4 “mandatory” trial domains: pain, function, patient global — shoulder, and adverse events including death; and 4 “important but optional” domains: participation (recreation/work), sleep, emotional well-being, and condition-specific pathophysiological manifestations. Cognitive dysfunction was voted out of the core domain set.Conclusion.OMERACT 2018 delegates endorsed a core domain set for clinical trials of shoulder disorders. The next step includes identification of a core outcome measurement set that passes the OMERACT 2.1 Filter for measuring each domain.

Summary Background The benefits and risks of testosterone treatment for women with diminished sexual wellbeing remain controversial. We did a systematic review and meta-analysis to assess potential benefits and risks of testosterone for women. Methods We searched MEDLINE, Embase, the Cochrane Central Register of Controlled Trials, and Web of Science for blinded, randomised controlled trials of testosterone treatment of at least 12 weeks' duration completed between Jan 1, 1990, and Dec 10, 2018. We also searched drug registration applications to the European Medicine Agency and the US Food and Drug Administration to identify any unpublished data. Primary outcomes were the effects of testosterone on sexual function, cardiometabolic variables, cognitive measures, and musculoskeletal health. This study is registered with the International Prospective Register of Systematic Reviews (PROSPERO), number CRD42018104073. Findings Our search strategy retrieved 46 reports of 36 randomised controlled trials comprising 8480 participants. Our meta-analysis showed that, compared with placebo or a comparator (eg, oestrogen, with or without progestogen), testosterone significantly increased sexual function, including satisfactory sexual event frequency (mean difference 0·85, 95% CI 0·52 to 1·18), sexual desire (standardised mean difference 0·36, 95% CI 0·22 to 0·50), pleasure (mean difference 6·86, 95% CI 5·19 to 8·52), arousal (standardised mean difference 0·28, 95% CI 0·21 to 0·35), orgasm (standardised mean difference 0·25, 95% CI 0·18 to 0·32), responsiveness (standardised mean difference 0·28, 95% CI 0·21 to 0·35), and self-image (mean difference 5·64, 95% CI 4·03 to 7·26), and reduced sexual concerns (mean difference 8·99, 95% CI 6·90 to 11·08) and distress (standardised mean difference −0·27, 95% CI −0·36 to −0·17) in postmenopausal women. A significant rise in the amount of LDL-cholesterol, and reductions in the amounts of total cholesterol, HDL-cholesterol, and triglycerides, were seen with testosterone administered orally, but not when administered non-orally (eg, by transdermal patch or cream). An overall increase in weight was recorded with testosterone treatment. No effects of testosterone were reported for body composition, musculoskeletal variables, or cognitive measures, although the number of women who contributed data for these outcomes was small. Testosterone was associated with a significantly greater likelihood of reporting acne and hair growth, but no serious adverse events were recorded. Interpretation Testosterone is effective for postmenopausal women with low sexual desire causing distress, with administration via non-oral routes (eg, transdermal application) preferred because of a neutral lipid profile. The effects of testosterone on individual wellbeing and musculoskeletal and cognitive health, as well as long-term safety, warrant further investigation. Funding Australian National Health and Medical Research Council.

ObjectiveTo examine the effect of regular breakfast consumption on weight change and energy intake in people living in high income countries.DesignSystematic review and meta-analysis.Data sourcesPubMed, Ovid Medline, and CINAHL were searched for randomised controlled trials published between January 1990 and January 2018 investigating the effect of breakfast on weight or energy intake. ClinicalTrials.gov and the World Health Organization’s International Clinical Trials Registry Platform search portal were also searched in October 2018 to identify any registered yet unpublished or ongoing trials.Eligibility criteria for selecting studiesRandomised controlled trials from high income countries in adults comparing breakfast consumption with no breakfast consumption that included a measure of body weight or energy intake. Two independent reviewers extracted the data and assessed the risk of bias of included studies. Random effects meta-analyses of the effect of breakfast consumption on weight and daily energy intake were performed.ResultsOf 13 included trials, seven examined the effect of eating breakfast on weight change, and 10 examined the effect on energy intake. Meta-analysis of the results found a small difference in weight favouring participants who skipped breakfast (mean difference 0.44 kg, 95% confidence interval 0.07 to 0.82), but there was some inconsistency across trial results (I2=43%). Participants assigned to breakfast had a higher total daily energy intake than those assigned to skip breakfast (mean difference 259.79 kcal/day, 78.87 to 440.71; 1 kcal=4.18 kJ), despite substantial inconsistency across trial results (I2=80%). All of the included trials were at high or unclear risk of bias in at least one domain and had only short term follow-ups (mean period seven weeks for weight, two weeks for energy intake). As the quality of the included studies was mostly low, the findings should be interpreted with caution.ConclusionThis study suggests that the addition of breakfast might not be a good strategy for weight loss, regardless of established breakfast habit. Caution is needed when recommending breakfast for weight loss in adults, as it could have the opposite effect. Further randomised controlled trials of high quality are needed to examine the role of breakfast eating in the approach to weight management.Study registrationPROSPERO registration number CRD42017057687.

This systematic review and meta-analysis evaluates the association of anorexia nervosa with the cancer incidence and mortality among study populations with anorexia nervosa compared with the general population or those without anorexia nervosa., Key Points Question Are people with anorexia nervosa at a higher risk of developing or dying of cancer compared with those without anorexia and the general population? Findings In this systematic review and meta-analysis of 7 cohort studies including more than 42 000 participants with anorexia nervosa, there was no association of anorexia nervosa with overall cancer incidence or mortality. Anorexia nervosa was inversely associated with breast cancer incidence but positively associated with risk of developing lung and esophageal cancer. Meaning There was no association of anorexia nervosa with risk of cancer overall and few associations of anorexia nervosa with risk of site-specific cancer., Importance Anorexia nervosa is recognized as an important cause of morbidity in young people. However, the risk of cancer in people with anorexia nervosa remains uncertain. Objective To evaluate the association of anorexia nervosa with the risk of developing or dying of cancer. Data Sources MEDLINE, Scopus, Embase, and Web of Science from database inception to January 9, 2019. Study Selection Published observational studies in humans examining the risk of cancer in people with anorexia nervosa compared with the general population or those without anorexia nervosa. Studies needed to report incidence or mortality rate ratios (RRs). Data Extraction and Synthesis Screening, data extraction, and methodological quality assessment were performed by at least 2 researchers independently. A random-effects model was used to synthesize individual studies. Heterogeneity (I2) was assessed and 95% prediction intervals (PIs) were calculated. Main Outcomes and Measures All cancer incidence and cancer mortality associated with anorexia nervosa. Secondary outcomes were site-specific cancer incidence and mortality. Results Seven cohort studies published in 10 articles (42 602 participants with anorexia nervosa) were included. Anorexia nervosa was not associated with risk of developing any cancer (4 studies in women; RR, 0.97; 95% CI, 0.89-1.06; P = .53; I2, 0%; 95% PI, 0.80-1.18; moderate confidence). Anorexia nervosa was associated with decreased breast cancer incidence (5 studies in women; RR, 0.60; 95% CI, 0.50-0.80; P

Objective This study aimed to evaluate the epidemiological and reporting characteristics of systematic reviews (SRs) in dentistry indexed within PubMed during the year 2017. Methods We searched for SRs in dentistry indexed within PubMed in 2017. Study selection was undertaken by two reviewers independently. Data related to epidemiological and reporting characteristics were extracted by one of three reviewers. A descriptive analysis of the data was performed. Characteristics of SRs were analyzed considering all SRs included and subgrouped by dental specialties. In addition, we explored if the reporting of 24 characteristics of treatment/therapeutic SRs was associated with the self-reported use of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Statement calculating the risk ratio (RR) with a 95% confidence interval for each characteristic. Results 495 articles fulfilled the eligibility criteria. The main specialty considered was Oral Surgery numbering 75 articles. Brazil presented the highest contribution with 117 SRs (23.6%). The reporting quality was variable. Items such as, use of the term “systematic review”, or “meta-analysis” in the title or abstract was well reported. In contrast, the study risk of bias/quality assessment method was not reported in 40.5% of SRs. In addition, only four reporting characteristics were described more often in those SR that reported using the PRISMA Statement. Conclusion A large number of SRs were published in dentistry in 2017 and the reporting and epidemiological characteristics varied among dental specialties. There is a mandatory need to improve the quality of reporting and conduct of SRs in dentistry. Clinical significance Poor reporting and conduction of SRs could generate SRs with imprecise and biased results.

Background: Funded health research is being published in journals that many regard as “predatory”, deceptive, and non-credible. We do not currently know whether funders provide guidance on how to select a journal in which to publish funded health research. Methods: We identified the largest 46 philanthropic, public, development assistance, public-private partnership, and multilateral funders of health research by expenditure, globally as well as four public funders from lower-middle income countries, from the list at https://healthresearchfunders.org. One of us identified guidance on disseminating funded research from each funders’ website (August/September 2017), then extracted information about selecting journals, which was verified by another assessor. Discrepancies were resolved by discussion. Results were summarized descriptively. This research used publicly available information; we did not seek verification with funding bodies. Results: The majority (44/50) of sampled funders indicated funding health research. 38 (of 44, 86%) had publicly available information about disseminating funded research, typically called “policies” (29, 76%). Of these 38, 36 (95%) mentioned journal publication for dissemination of which 13 (36.11%) offer variable guidance on selecting a journal, all of which relate to the funder’s open access mandate. Six funders (17%) outlined publisher requirements or features by which to select a journal. One funder linked to a document providing features of journals to look for (e.g. listed in the Directory of Open Access Journals) and to be wary of (e.g., no journal scope statement, uses direct and unsolicited marketing). Conclusions: Few funders provided guidance on how to select a journal in which to publish funded research. Funders have a duty to ensure that the research they fund is discoverable by others. This research is a benchmark for funder guidance on journal selection prior to the January 2021 implementation of Plan S (a global, funder-led initiative to ensure immediate, open access to funded, published research).

Objective To determine whether physiotherapists avoid lumbar X-rays for acute non-specific low back pain and advise people to stay active. Methods We conducted a cross sectional survey of Australian physiotherapists. 880 physiotherapists were randomly sampled from Victoria (495), South Australia (158), and Western Australia (227). Physiotherapists were asked which investigations they would order and interventions they would provide for five acute low back pain (LBP) presentations described in vignettes. Four of the five vignettes represented people who would not require a plain lumbar X-ray and would benefit from advice to stay active; one described a patient with a suspected vertebral fracture and would require a plain X-ray. Participants selected from a list of response options or provided free text responses. Results Questionnaires were completed by 203 of 567 potentially eligible physiotherapists (response rate 36%). Across the four vignettes where an X-ray was not indicated, 75% (95%CI 71–78%) of physiotherapists reported they would practice concordant with the guidelines and not order an X-ray, and 62% (95%CI 57–66%) provided advice to stay active. Conclusions Most physiotherapists report intended compliance with recommendations in Australian clinical practice guidelines (CPGs) regarding avoiding the use of X-rays and providing advice to stay active for people with simple acute low back pain, given a vignette based scenario. The majority of respondents reported that they would not advise bed rest. Possible opportunities to further enhance compliance need to be developed and tested to reinforce the role of CPGs in informing physiotherapy practice.

IntroductionCurrent recommendations for vitamin D and calcium in dietary guidelines and bone health guidelines vary significantly among countries and professional organisations. It is unknown whether the methods used to develop these recommendations followed a rigourous process and how the differences in methods used may affect the recommended intakes of vitamin D and calcium. The objectives of this study are (1) collate and compare recommendations for vitamin D and calcium across guidelines, (2) appraise methodological quality of the guideline recommendations and (3) identify methodological factors that may affect the recommended intakes for vitamin D and calcium. This study will make a significant contribution to enhancing the methodological rigour in public health guidelines for vitamin D and calcium recommendations.Methods and analysesWe will conduct a systematic review to evaluate vitamin D and calcium recommendations for osteoporosis prevention in generally healthy middle-aged and older adults. Methodological assessment will be performed for each guideline against those outlined in the 2014 WHO handbook for guideline development. A systematic search strategy will be applied to locate food-based dietary guidelines and bone health guidelines indexed in various electronic databases, guideline repositories and grey literature from 1 January 2009 to 28 February 2019. Descriptive statistics will be used to summarise the data on intake recommendation and on proportion of guidelines consistent with the WHO criteria. Logistic regression, if feasible, will be used to assess the relationships between the methodological factors and the recommendation intakes.Ethics and disseminationEthics approval is not required as we will only extract published data or information from the published guidelines. Results of this review will be disseminated through conference presentations and peer-reviewed publications.PROSPERO registration numberCRD42019126452

BackgroundSeveral scales, checklists and domain-based tools for assessing risk of reporting biases exist, but it is unclear how much they vary in content and guidance. We conducted a systematic review of the content and measurement properties of such tools.MethodsWe searched for potentially relevant articles in Ovid MEDLINE, Ovid Embase, Ovid PsycINFO and Google Scholar from inception to February 2017. One author screened all titles, abstracts and full text articles, and collected data on tool characteristics.ResultsWe identified 18 tools that include an assessment of the risk of reporting bias. Tools varied in regard to the type of reporting bias assessed (eg, bias due to selective publication, bias due to selective non-reporting), and the level of assessment (eg, for the study as a whole, a particular result within a study or a particular synthesis of studies). Various criteria are used across tools to designate a synthesis as being at ‘high’ risk of bias due to selective publication (eg, evidence of funnel plot asymmetry, use of non-comprehensive searches). However, the relative weight assigned to each criterion in the overall judgement is unclear for most of these tools. Tools for assessing risk of bias due to selective non-reporting guide users to assess a study, or an outcome within a study, as ‘high’ risk of bias if no results are reported for an outcome. However, assessing the corresponding risk of bias in a synthesis that is missing the non-reported outcomes is outside the scope of most of these tools. Inter-rater agreement estimates were available for five tools.ConclusionThere are several limitations of existing tools for assessing risk of reporting biases, in terms of their scope, guidance for reaching risk of bias judgements and measurement properties. Development and evaluation of a new, comprehensive tool could help overcome present limitations.

Background The International Prospective Register of Systematic Reviews (PROSPERO) was launched in February 2011 to increase transparency of systematic reviews (SRs). There have been few investigations of the content and use of the database. We aimed to investigate the number of PROSPERO registrations from inception to 2017, and website usage in the last year. We also aimed to explore the epidemiological characteristics of and completeness of primary outcome pre-specification in a sample of PROSPERO records from 2017. Methods The PROSPERO database managers provided us with data on the annual and cumulative number of SR registrations up to October 10, 2017, and the number of visits to the PROSPERO website over the year preceding October 10, 2017. One author collected data on the focus of the SR (e.g. therapeutic, diagnostic), health area addressed, funding source and completeness of outcome pre-specification in a random sample of 150 records of SRs registered in PROSPERO between April 1, 2017 and September 30, 2017. Results As of October 10, 2017, there were 26,535 SRs registered in PROSPERO; guided by current monthly submission rates, we anticipate this figure will reach over 30,000 by the end of 2017. There has been a 10-fold increase in registrations, from 63 SRs per month in 2012 to 800 per month in 2017. In the year preceding October 10, 2017, the PROSPERO website received more than 1.75 million page views. In the random sample of 150 registered SRs, the majority were focused on a therapeutic question (78/150 [52%]), while only a few focused on a diagnostic/prognostic question (11/150 [7%]). The 150 registered SRs addressed 18 different health areas. Any information about the primary outcome other than the domain (e.g. timing, effect measures) was not pre-specified in 44/150 records (29%). Conclusions Registration of SRs in PROSPERO increased rapidly between 2011 and 2017, thus benefiting users of health evidence who want to know about ongoing SRs. Further work is needed to explore how closely published SRs adhere to the planned methods, whether greater pre-specification of outcomes prevents selective inclusion and reporting of study results, and whether registered SRs address necessary questions.

Systematic reviews often encounter primary studies that report multiple effect sizes based on data from the same participants. These have the potential to introduce statistical dependency into the meta-analytic data set. In this paper, we provide a tutorial on dealing with effect size multiplicity within studies in the context of meta-analyses of intervention and association studies, recommending a three-step approach. The first step is to define the research question and consider the extent to which it mainly reflects interest in mean effect sizes (which we term a convergent approach) or an interest in exploring heterogeneity (which we term a divergent approach). A second step is to identify the types of multiplicities that appear in the initial database of effect sizes relevant to the research question, and we propose a categorization scheme to differentiate them. The third step is to select a strategy for dealing with each type of multiplicity. The researcher can choose between a reductionist meta-analytic approach, which is characterized by inclusion of a single effect size per study, and an integrative approach, characterized by inclusion of multiple effect sizes per study. We present an overview of available analysis strategies for dealing with effect size multiplicity within studies and provide recommendations intended to help researchers decide which strategy might be preferable in particular situations. Last, we offer caveats and cautions about addressing the challenges multiplicity poses for systematic reviews and meta-analyses.

ObjectivesTo explore the outcome domains and measurement instruments reported across randomized trials of any interventions for various shoulder disorders.MethodsWe searched for shoulder trials included in Cochrane reviews published up to Issue 10, 2015, or indexed in PubMed between 2006 and 2015. Trials were eligible for inclusion if they focused on any intervention for rotator cuff disease, adhesive capsulitis, shoulder instability, glenohumeral or acromioclavicular osteoarthritis, shoulder dislocation, proximal humeral or humeral head fractures, or unspecified shoulder pain. Two authors independently selected trials for inclusion and extracted information on the domains and measurement instruments reported, with consensus discussion among all authors where required.ResultsWe included 409 trials, published between 1954 and 2015. Across the trials, we identified 319 different instruments that were classified into 32 domains. Most trials reported a measure of pain (90%), range of motion (78%), and physical function (71%). Measurement of adverse events was reported in only 31% of trials. Muscle strength was reported in 44% of trials and imaging outcomes were reported in 21% of trials. Other patient-reported outcome measures such as global assessment of treatment success, health-related quality of life, work ability, and psychological functioning, were each reported in 15% or fewer trials. Most domains were reported at a similar frequency across different shoulder disorders.ConclusionThere was wide diversity in the domains and measurement instruments reported. Our results provide the foundation for the development of a core outcome set for use in future trials across all shoulder disorders.

Methods to assess the risk of bias in a way that is reliable, reproducible. and transparent to readers, have evolved over time. Viswanathan et al. recently provided updated recommendations for assessing risk of bias in systematic reviews of health care interventions. We comment on their recommendations and discuss new tools in development that we, as co-convenors and coordinators of the Cochrane Bias Methods Group, are leading, which complement the methods recommended.

[Background]: The management of comorbidity and multimorbidity poses major challenges to health services around the world. Analysis of scientific research in comorbidity and multimorbidity is limited in the biomedical literature. This study aimed to map global scientific research in comorbidity and multimorbidity to understand the maturity and growth of the area during the past decades. [Methods and findings]: This was a cross-sectional analysis of the Web of Science. Searches were run from inception until November 8, 2016. We included research articles or reviews with no restrictions by language or publication date. Data abstraction was done by one researcher. A process of standardization was conducted by two researchers to unify different terms and grammatical variants and to remove typographical, transcription, and/or indexing errors. All potential discrepancies were resolved via discussion. Descriptive analyses were conducted (including the number of papers, citations, signatures, most prolific authors, countries, journals and keywords). Network analyses of collaborations between countries and co-words were presented. During the period 1970–2016, 85994 papers (64.0% in 2010–2016) were published in 3500 journals. There was wide diversity in the specialty of the journals, with psychiatry (16558 papers; 19.3%), surgery (9570 papers; 11.1%), clinical neurology (9275 papers; 10.8%), and general and internal medicine (7622 papers; 8.9%) the most common. PLOS One (1223 papers; 1.4%), the Journal of Affective Disorders (1154 papers; 1.3%), the Journal of Clinical Psychiatry (727 papers; 0.8%), the Journal of the American Geriatrics Society (634 papers; 0.7%) and Obesity Surgery (588 papers; 0.7%) published the largest number of papers. 168 countries were involved in the production of papers. The global productivity ranking was headed by the United States (37624 papers), followed by the United Kingdom (7355 papers), Germany (6899 papers) and Canada (5706 papers). Twenty authors who published 100 or more papers were identified; the most prolific authors were affiliated with Harvard Medical School, State University of New York Upstate Medical University, National Taiwan Normal University and China Medical University. The 50 most cited papers (“citation classics” with at least 1000 citations) were published in 20 journals, led by JAMA Psychiatry (11 papers) and JAMA (10 papers). The most cited papers provided contributions focusing on methodological aspects (e.g. Charlson Comorbidity Index, Elixhauser Comorbidity Index, APACHE prognostic system), but also important studies on chronic diseases (e.g. epidemiology of mental disorders and its correlates by the U.S. National Comorbidity Survey, Fried’s frailty phenotype or the management of obesity). [Conclusions]: Ours is the first analysis of global scientific research in comorbidity and multimorbidity. Scientific production in the field is increasing worldwide with research leadership of Western countries, most notably, the United States., FC-L and RT-S are funded by the Generalitat Valenciana (PROMETEOII/2015/021) and CIBERSAM. RT-S is partially funded by grant PIE14/00031 from the Institute of Health Carlos III (ISCIII) and co-funded by the European Regional Development Fund (ERDF). AA-A and RA-B are funded by the Spanish Ministry of Economy and Competitiveness (CSO2015-65594-C2-2-R). MJP is supported by an Australian National Health and Medical Research Council Early Career Fellowship (1088535). BH is supported by a New Investigator Award from the Canadian Institutes of Health Research and the Drug Safety and Effectiveness Network.

Objective.The Outcome Measures in Rheumatology (OMERACT) Shoulder Core Outcome Set Special Interest Group (SIG) was established to develop a core outcome set (COS) for clinical trials of shoulder disorders.Methods.In preparation for OMERACT 2016, we systematically examined all outcome domains and measurement instruments reported in 409 randomized trials of interventions for shoulder disorders published between 1954 and 2015. Informed by these data, we conducted an international Delphi consensus study including shoulder trial experts, clinicians, and patients to identify key domains that should be included in a shoulder disorder COS. Findings were discussed at a stakeholder premeeting of OMERACT. At OMERACT 2016, we sought consensus on a preliminary core domain set and input into next steps.Results.There were 13 and 15 participants at the premeeting and the OMERACT 2016 SIG meeting, respectively (9 attended both meetings). Consensus was reached on a preliminary core domain set consisting of an inner core of 4 domains: pain, physical function/activity, global perceived effect, and adverse events including death. A middle core consisted of 3 domains: emotional well-being, sleep, and participation (recreation and work). An outer core of research required to inform the final COS was also formulated.Conclusion.Our next steps are to (1) analyze whether participation (recreation and work) should be in the inner core, (2) conduct a third Delphi round to finalize definitions and wording of domains and reach final endorsement for the domains, and (3) determine which instruments fulfill the OMERACT criteria for measuring each domain.

Background The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations. Methods We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated). Results We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions. Conclusions Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs. Electronic supplementary material The online version of this article (10.1186/s13643-017-0663-8) contains supplementary material, which is available to authorized users.

Predatory journals have shoddy reporting and include papers from wealthy nations, find David Moher, Larissa Shamseer, Kelly Cobey and colleagues.

Objectives To investigate the application and interpretation of statistical analyses in a cross-section of systematic reviews (SRs) of therapeutic interventions, without restriction by journal, clinical condition, or specialty. Study Design and Setting We evaluated a random sample of SRs assembled previously, which were indexed in MEDLINE® during February 2014, focused on a treatment or prevention question, and reported at least one meta-analysis. The reported statistical methods used in each SR were extracted from articles and online appendices by one author, with a 20% random sample extracted in duplicate. Results We evaluated 110 SRs; 78/110 (71%) were non-Cochrane SRs, and 55/110 (50%) investigated a pharmacological intervention. The SRs presented a median of 13 (interquartile range 5-27) meta-analytic effects. When considering the index (primary or first reported) meta-analysis of each SR, just over half (62/110 [56%]) used the random-effects model, but few (5/62 [8%]) interpreted the meta-analytic effect correctly (as the average of the intervention effects across all studies). A statistical test for funnel plot asymmetry was reported in 17/110 (15%) SRs, however, in only 4/17 (24%) did the test include the recommended number of at least 10 studies of varying size. Subgroup analyses accompanied 42/110 (38%) index meta-analyses, but findings were not interpreted with respect to a test for interaction in 29/42 (69%) cases, and the issue of potential confounding in the subgroup analyses was not raised in any SR. Conclusions There is scope for improvement in the application and interpretation of statistical analyses in SRs of therapeutic interventions. Involvement of statisticians on the SR team, and establishment of partnerships between researchers with specialist expertise in SR methods and journal editors may help overcome these shortcomings.

Objectives The aim of the study was to characterize methodological conduct, reporting, and quality of five knowledge synthesis (KS) approaches. Study Design and Setting Retrospective analysis of a convenience sample of five published databases of KS approaches: overview of reviews ( n = 74), scoping reviews ( n = 494), rapid reviews ( n = 84), systematic reviews ( n = 300), and network meta-analyses (NMAs; n = 456). Data in the five published databases were abstracted by two reviewers independently, any missing data for this retrospective analysis were abstracted by one experienced reviewer. Methods were appraised using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR) tool. Descriptive analysis was performed. Results Reporting the use of a protocol ranged from 4% for rapid reviews to 32% for systematic reviews. The use of two reviewers for citation and full-text screening ranged from 20% for scoping reviews to 60% for NMAs. Data abstraction was performed in duplicate for 11% of rapid reviews and 54% of NMAs, and for risk of bias appraisal, this ranged from 6% for scoping reviews to 41% for NMAs. NMAs had the highest median percentage of maximum obtainable AMSTAR score (64%; Q1–Q3:45–73%), while scoping reviews had the lowest (25%; Q1–Q3:13–38%). Conclusion NMAs consistently scored the highest on the AMSTAR tool likely because the purpose is to estimate treatment effects statistically. Scoping reviews scored the lowest (even after adjusting the score for not relevant items) likely because the purpose is to characterize the literature.