Search

Background: Protocols of systematic reviews allow for planning and documentation of review methods and thus improve the transparency of the reviews process. However, pre-registration of a protocol is not enough, the author also need to follow it. PROSPERO is an open-access online database for the registration of non-Cochrane systematic reviews. The purpose of this study is to compare published non-Cochrane reviews with their pre-registered protocols on PROSPERO to determine what changes, if any, have been made, and how likely these changes are to impact the quality of systematic review. Methods: This is a retrospective comparative study. We searched for protocols on PROSPERO platform that were registered in 2018 and then selected the protocols that full text have been published as of January 1st 2019. Published full texts were identified through the protocol's final publication citation. Two authors independently compared and identified changes between protocols and systematic reviews and then evaluated the impact (improve, reduce, or unclear) of these changes on the reporting or methodology quality of reviews. Descriptive statistics of percentage (%) and frequency (n) were conducted. Results: We identified 39 pairs, all of which exhibited changes. “Search strategy”(92%, n=36), “data extraction”(90%, n=35), “data synthesis”(77%, n=30), “outcome”(64%, n=24), and “subgroup analysis”(64%, n=24) all showed significant changes. All changes to only one review were considered to improve the reporting or methodology quality, and the remaining 97% of reviews (n=38) contained changes that were considered to reduce the methodology or reporting quality or that had an unclear impact on systematic reviews. Conclusions: Changes between the non-Cochrane systematic reviews and their protocols recorded on PROSPERO were widespread. Some of the changes reduced the methodology or reporting quality of systematic reviews or had an unclear impact. Measures should be taken to further improve the transparency of the non-Cochrane systematic reviews. Adding a new item in updated “Preferred Reporting Items for Systematic reviews and Meta-Analyses” (PRISMA) and “Meta-analysis of Observational Studies in Epidemiology” (MOOSE) to guide reporting and explaining the changes, as well as advising peer reviewers (and editors) to check the reviews against the protocols are two suggested fundamental solutions.

ObjectiveTo assess the methodological quality of animal studies published in China and abroad using the SYRCLE’s risk of bias tool, and to provide references to improve the methodological quality of animal studies to encourage high quality preclinical studies.MethodsAn electronic search was performed in the Chinese Scientific Citation Database (CSCD) and Web of Science from 2014 to October 2018. Document screening and data extraction were performed independently by four researchers. The methodological quality of the included studies was assessed using the SYRCLE’s risk of bias tool. Statistical analysis was performed using SPSS23.ResultsA total of 2764 animal studies were included. Of the studies, 984 were published in English and 1780 were in Chinese. The citation frequency of more than 90% of the included studies was less than 5. The results of methodological quality assessment showed that 36.36% (8/22) of the sub-items were rated as “low risk” in more than 50% of the included studies, of which 75% (6/8) were rated as “low risk” in more than 80% of the included studies. A total of 59.09% (13/22) of the sub-items were rated as “low risk” in less than 30% of the included studies, of which 92.31% (12/13) were rated as “low risk” in less than 10% of the included studies. The incidence of “low risk” Chinese studies regarding performance bias, detection bias and reporting bias were lower than English studies. For foreign studies, more attention should be paid to selection bias, attrition bias, and reporting bias.ConclusionWe identified limitations in the methodological quality of animal experiment studies published in China and abroad. We therefore suggest that it is necessary to take targeted measures to popularize the SYRCLE’s risk of bias tool to effectively improve the design and implementation of animal experiments, and guide study development.