Scoping review of COVID-19-related systematic reviews and meta-analyses: can we really have confidence in their results?

Aim: The aim of this study was to systematically appraise the quality of a sample of COVID-19-related systematic reviews (SRs) and discuss internal validity threats affecting the COVID-19 body of evidence.
Design: We conducted a scoping review of the literature. SRs with or without meta-analysis (MA) that evaluated clinical data, outcomes or treatments for patients with COVID-19 were included.
Main Outcome Measures: We extracted quality characteristics guided by A Measurement Tool to Assess Systematic Reviews-2 to calculate a qualitative score. Complementary evaluation of the most prominent published limitations affecting the COVID-19 body of evidence was performed.
Results: A total of 63 SRs were included. The majority were judged as a critically low methodological quality. Most of the studies were not guided by a pre-established protocol (39, 62%). More than half (39, 62%) failed to address risk of bias when interpreting their results. A comprehensive literature search strategy was reported in most SRs (54, 86%). Appropriate use of statistical methods was evident in nearly all SRs with MAs (39, 95%). Only 16 (33%) studies recognised heterogeneity in the definition of severe COVID-19 as a limitation of the study, and 15 (24%) recognised repeated patient populations as a limitation.
Conclusion: The methodological and reporting quality of current COVID-19 SR is far from optimal. In addition, most of the current SRs fail to address relevant threats to their internal validity, including repeated patients and heterogeneity in the definition of severe COVID-19. Adherence to proper study design and peer-review practices must remain to mitigate current limitations.
Competing Interests: Competing interests: Nothing to disclose related to the work described in this article. The laboratory of CAS holds patents on the function of PRKAR1A, PDE11A and GPR101 molecules and has received research funding from Pfizer for work related to GPR101 and acromegaly/gigantism. The funders had no role in the design and conduct of this study, or the preparation of this manuscript.
(© Author(s) (or their employer(s)) 2022. No commercial re-use. See rights and permissions. Published by BMJ.)