Search

We read with considerable interest the study by Gusenbauer and Haddaway (Gusenbauer and Haddaway, 2020, Research Synthesis Methods, doi:10.1002/jrsm.1378) comparing the systematic search qualities of 28 search systems, including Google Scholar (GS) and PubMed. Google Scholar and PubMed are the two most popular free academic search tools in biology and chemistry, with GS being the number one search tool in the world. Those academics using GS as their principal system for literature searches may be unaware of research which enumerates five critical features for scientific literature tools that greatly influenced Gusenbauer's 2020 study. Using this list as the framework for a targeted comparison between just GS and PubMed, we found stark differences which overwhelmingly favored PubMed. In this comment, we show that by comparing the characteristics of the two search tools, features that are particularly useful in one search tool, but are missing in the other, are strikingly spotlighted. One especially popular feature that ubiquitously appears in GS, but not in PubMed, is the forward citation search found under every citation as a clickable Cited by N link. We seek to improve the PubMed search experience using two approaches. First, we request that PubMed add Cited by N links, making them as omnipresent as the GS links. Second, we created an open‐source command‐line tool, pmidcite, which is used alongside PubMed to give information to researchers to help with the choice of the next paper to examine, analogous to how GS's Cited by N links help to guide users. Find pmidcite at https://github.com/dvklopfenstein/pmidcite. [ABSTRACT FROM AUTHOR]

Background: When literature searching for systematic reviews, it is good practice to search widely across different information sources. Little is known about the contributions of different publication formats (e.g. journal article and book chapter) and sources, especially for studies of people's views. Method: Studies from four reviews spanning three public health areas (active transport, motherhood and obesity) were analysed in terms of publication formats and the information sources they were identified from. They comprised of 229 studies exploring people's perceptions, beliefs and experiences ("views studies") and were largely qualitative. Results: Although most (61%) research studies were published within journals, nearly a third (29%) were published as research reports and 5% were published in books. The remainder consisted of theses, conference papers and raw datasets. Two-thirds of studies (66%) were located in a total of 19 bibliographic databases, and 15 databases provided studies that were not identified elsewhere. PubMed was a good source for all reviews. Supplementary information sources were important for identifying studies in all publication formats. Conclusions: Undertaking sensitive searches across a range of information sources is essential for locating views studies in all publication formats. We discuss some benefits and challenges of utilising different information sources.

Reviewing complex interventions is challenging because they include many elements that can interact dynamically in a nonlinear manner. A systems perspective offers a way of thinking to help understand complex issues, but its application in evidence synthesis is not established. The aim of this project was to understand how and why systems perspectives have been applied in evidence synthesis. A methodological mapping review was conducted to identify papers using a systems perspective in evidence synthesis. A search was conducted in seven bibliographic databases and three search engines. A total of 101 papers (representing 98 reviews) met the eligibility criteria. Two categories of reviews were identified: (1) reviews using a "systems lens" to frame the topic, generate hypotheses, select studies, and guide the analysis and interpretation of findings (n = 76) and (2) reviews using systems methods to develop a systems model (n = 22). Several methods (e.g., systems dynamic modeling, soft systems approach) were identified, and they were used to identify, rank and select elements, analyze interactions, develop models, and forecast needs. The main reasons for using a systems perspective were to address complexity, view the problem as a whole, and understand the interrelationships between the elements. Several challenges for capturing the true nature and complexity of a problem were raised when performing these methods. This review is a useful starting point when designing evidence synthesis of complex interventions. It identifies different opportunities for applying a systems perspective in evidence synthesis, and highlights both commonplace and less familiar methods. [ABSTRACT FROM AUTHOR]

Guidelines produced by local, national and international bodies underpin clinical practice and healthcare services worldwide. For guidelines to be based on the best available evidence, it is critical that syntheses of both qualitative and quantitative evidence are used to inform decision‐making. As methods for qualitative evidence syntheses (QES) develop, they are increasingly able to inform health guideline production. However, the process whereby this form of evidence is considered and incorporated tends to be unclear. This systematic review synthesized existing guidance concerning the use of QES in guideline development. Sources published in English that described or prescribed methods for incorporating QES into evidence‐based health guidelines were eligible for inclusion. Seventeen relevant papers were identified. The literature indicates that there is a reasonable consensus about many stages of conducting a QES to inform guideline development. Areas needing further exploration include: the way that committees engage with QES; the usefulness of different QES methodologies; and understanding of how expert committees use evidence. Methods for producing QES for guideline committees tend to be similar to quantitative systematic review methods in terms of searching, quality appraisal, systematic management of data, and presentation of results. While this allows transparency and accountability, it could be argued that it is less "true" to the principles of being led by the data, which are fundamental to most qualitative research. Understanding the process of using QES to produce guidelines is critical to determining their validity and applicability, and to ensure that healthcare provision is based on the best available evidence. [ABSTRACT FROM AUTHOR]

Objective: This paper describes several automation tools and software that can be considered during evidence synthesis projects and provides guidance for their integration in the conduct of scoping reviews. Study Design and Setting: The guidance presented in this work is adapted from the results of a scoping review and consultations with the JBI Scoping Review Methodology group. Results: This paper describes several reliable, validated automation tools and software that can be used to enhance the conduct of scoping reviews. Developments in the automation of systematic reviews, and more recently scoping reviews, are continuously evolving. We detail several helpful tools in order of the key steps recommended by the JBI's methodological guidance for undertaking scoping reviews including team establishment, protocol development, searching, de‐duplication, screening titles and abstracts, data extraction, data charting, and report writing. While we include several reliable tools and software that can be used for the automation of scoping reviews, there are some limitations to the tools mentioned. For example, some are available in English only and their lack of integration with other tools results in limited interoperability. Conclusion: This paper highlighted several useful automation tools and software programs to use in undertaking each step of a scoping review. This guidance has the potential to inform collaborative efforts aiming at the development of evidence informed, integrated automation tools and software packages for enhancing the conduct of high‐quality scoping reviews. [ABSTRACT FROM AUTHOR]

We explored the value of handsearching to identify diagnostic test accuracy (DTA) studies to inform systematic reviews of DTA. Handsearching was conducted alongside a systematic review of the DTA of 18F-fluorodeoxyglucose positron emission tomography-computed tomography (FDG PET-CT) for colorectal cancer. Ten journals, identified by frequency analysis of studies from six imaging reviews, were handsearched for reports of DTA studies. The numbers of studies identified by handsearching and by database searching were compared. A total of 573 journal issues from ten journals were handsearched in 185 h. A total of 936 potential reports of DTA studies were identified: 25 were relevant to the FDG PET-CT review. 7/25 FDG PET-CT papers had not been identified by database searches. No papers met the systematic review inclusion criteria. The FDG-PET systematic review included 30 papers, from 24 different journals. Handsearching two of those journals identified 211 potential reports of DTA studies for all topics and 18 for FDG PET-CT. Handsearching identified previously unseen papers but did not yield unique relevant DTA studies for the specific review. DTA imaging studies are widely distributed, and it may be more efficient to choose journals to handsearch after the identification of some relevant studies. Copyright © 2012 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

The involvement of non‐researcher contributors (eg, stakeholders, patients and the public, decision and policy makers, experts, lay contributors) has taken a variety of forms within evidence syntheses. Realist reviews are a form of evidence synthesis that involves non‐researcher contributors yet this practice has received little attention. In particular, the role of patient and public involvement (PPI) has not been clearly documented. This review of reviews describes the ways in which contributor involvement, including PPI, is documented within healthcare realist reviews published over the last five years. A total of 448 papers published between 2014 and 2019 were screened, yielding 71 full‐text papers included in this review. Statements about contributor involvement were synthesized across each review using framework analysis. Three themes are described in this article including nomenclature, nature of involvement, and reporting impact. Papers indicate that contributor involvement in realist reviews refers to stakeholders, experts, or advisory groups (ie, professionals, clinicians, or academics). Patients and the public are occasionally subsumed into these groups and in doing so, the nature and impact of their involvement become challenging to identify and at times, is lost completely. Our review findings indicate a need for the realist review community to develop guidance to support researchers in their future collaboration with contributors, including patients and the public. [ABSTRACT FROM AUTHOR]

Extracting data from studies is the norm in meta‐analyses, enabling researchers to generate effect sizes when raw data are otherwise not available. While there has been a general push for increased reproducibility in meta‐analysis, the transparency and reproducibility of the data extraction phase is still lagging behind. Unfortunately, there is little guidance of how to make this process more transparent and shareable. To address this, we provide several steps to help increase the reproducibility of data extraction in meta‐analysis. We also provide suggestions of R software that can further help with reproducible data policies: the shinyDigitise and juicr packages. Adopting the guiding principles listed here and using the appropriate software will provide a more transparent form of data extraction in meta‐analyses. [ABSTRACT FROM AUTHOR]

Systematic reviews and maps are considered a reliable form of research evidence, but often neglect non-English-language literature, which can be a source of important evidence. To understand the barriers that might limit authors' ability or intent to find and include non-English-language literature, we assessed factors that may predict the inclusion of non-English-language literature in ecological systematic reviews and maps, as well as the review authors' perspectives. We assessed systematic reviews and maps published in Environmental Evidence (n = 72). We also surveyed authors from each paper (n = 32 responses), gathering information on the barriers to the inclusion of non-English language literature. 44% of the reviewed papers (32/72) excluded non-English literature from their searches and inclusions. Commonly cited reasons included constraints related to resources and time. Regression analysis revealed that reviews with larger author teams, authors from diverse countries, especially those with non-English primary languages, and teams with multilingual capabilities searched in a significantly greater number of non-English languages. Our survey exposed limited language diversity within the review teams and inadequate funding as the principal barriers to incorporating non-English language literature. To improve language inclusion and reduce bias in systematic reviews and maps, our study suggests increasing language diversity within review teams. Combining machine translation with language skills can alleviate the financial and resource burdens of translation. Funding applications could also include translation costs. Additionally, establishing language exchange systems would enable access to information in more languages. Further studies investigating language inclusion in other journals would strengthen these conclusions. [ABSTRACT FROM AUTHOR]

Conventional systematic reviews offer few insights into for whom and how interventions work. 'Realist reviews' examine such questions via examining 'context‐mechanism‐outcome configurations' (CMOCs) but are insufficiently rigorous in how evidence is identified, assessed and synthesised. We developed 'realist systematic reviews', addressing similar questions to realist reviews but using rigorous methods. We applied this to synthesising evidence on school‐based prevention of dating and relationship violence (DRV) and gender‐based violence (GBV). This paper reflects on overall methods and findings, drawing on papers reporting each analysis. Drawing on intervention descriptions, theories of change and process evaluations, we developed initial CMOC hypotheses: interventions triggering 'school‐transformation' mechanisms (preventing violence by changing school environments) will achieve larger effects than those triggering 'basic‐safety' (stopping violence by emphasising its unacceptability) or 'positive‐development' (developing students' broader skills and relationships) mechanisms; however, school transformation would only work in schools with high organisational capacity. We used various innovative analyses, some of which aimed to test these hypotheses and some of which were inductive, drawing on available findings to augment and refine the CMOCs. Overall, interventions were effective in reducing long‐term DRV but not GBV or short‐term DRV. DRV prevention occurred most effectively via the 'basic‐safety' mechanism. 'School‐transformation' mechanisms were more effective in preventing GBV but only in high‐income countries. Impacts on long‐term DRV victimisation were greater when working with a critical mass of participating girls. Impacts on long‐term DRV perpetration were greater for boys. Interventions were more effective when focusing on skills, attitudes and relationships, or lacking parental involvement or victim stories. Our method provided novel insights and should be useful to policy‐makers seeking the best interventions for their contexts and the most information to inform implementation. [ABSTRACT FROM AUTHOR]

Objective: This paper describes several automation tools and software that can be considered during evidence synthesis projects and provides guidance for their integration in the conduct of scoping reviews. Study Design and Setting: The guidance presented in this work is adapted from the results of a scoping review and consultations with the JBI Scoping Review Methodology group. Results: This paper describes several reliable, validated automation tools and software that can be used to enhance the conduct of scoping reviews. Developments in the automation of systematic reviews, and more recently scoping reviews, are continuously evolving. We detail several helpful tools in order of the key steps recommended by the JBI's methodological guidance for undertaking scoping reviews including team establishment, protocol development, searching, de-duplication, screening titles and abstracts, data extraction, data charting, and report writing. While we include several reliable tools and software that can be used for the automation of scoping reviews, there are some limitations to the tools mentioned. For example, some are available in English only and their lack of integration with other tools results in limited interoperability. Conclusion: This paper highlighted several useful automation tools and software programs to use in undertaking each step of a scoping review. This guidance has the potential to inform collaborative efforts aiming at the development of evidence informed, integrated automation tools and software packages for enhancing the conduct of high-quality scoping reviews.

The paper by Lee and Beretvas (doi:10.1002/jrsm.1585) described a well‐executed simulation study comparing 'modern' with 'ad hoc' methods for performing meta‐regression when some covariates are incomplete. However, they drew practical conclusions after simulating data under a single missing data mechanism which favoured the 'modern' methods, while other missing data mechanisms would have favoured the 'ad hoc' methods. Broad recommendations about methods to use in practice should instead be based on simulation studies using a range of plausible data‐generating mechanisms. This range must represent what is believed likely to occur in practice, and not what is convenient for statistical analysis. [ABSTRACT FROM AUTHOR]

The amount of grey literature and 'softer' intelligence from social media or websites is vast. Given the long lead‐times of producing high‐quality peer‐reviewed health information, this is causing a demand for new ways to provide prompt input for secondary research. To our knowledge, this is the first review of automated data extraction methods or tools for health‐related grey literature and soft data, with a focus on (semi)automating horizon scans, health technology assessments (HTA), evidence maps, or other literature reviews. We searched six databases to cover both health‐ and computer‐science literature. After deduplication, 10% of the search results were screened by two reviewers, the remainder was single‐screened up to an estimated 95% sensitivity; screening was stopped early after screening an additional 1000 results with no new includes. All full texts were retrieved, screened, and extracted by a single reviewer and 10% were checked in duplicate. We included 84 papers covering automation for health‐related social media, internet fora, news, patents, government agencies and charities, or trial registers. From each paper, we extracted data about important functionalities for users of the tool or method; information about the level of support and reliability; and about practical challenges and research gaps. Poor availability of code, data, and usable tools leads to low transparency regarding performance and duplication of work. Financial implications, scalability, integration into downstream workflows, and meaningful evaluations should be carefully planned before starting to develop a tool, given the vast amounts of data and opportunities those tools offer to expedite research. [ABSTRACT FROM AUTHOR]

In any meta‐analysis, it is critically important to report the dispersion in effects as well as the mean effect. If an intervention has a moderate clinical impact on average we also need to know if the impact is moderate for all relevant populations, or if it varies from trivial in some to major in others. Or indeed, if the intervention is beneficial in some cases but harmful in others. Researchers typically report a series of statistics such as the Q‐value, the p‐value, and I2, which are intended to address this issue. Often, they use these statistics to classify the heterogeneity as being low, moderate, or high and then use these classifications when considering the potential utility of the intervention. While this practice is ubiquitous, it is nevertheless incorrect. The statistics mentioned above do not actually tell us how much the effect size varies. Classifications of heterogeneity based on these statistics are uninformative at best, and often misleading. My goal in this paper is to explain what these statistics do tell us, and that none of them tells us how much the effect size varies. Then I will introduce the prediction interval, the statistic that does tell us how much the effect size varies, and that addresses the question we have in mind when we ask about heterogeneity. This paper is adapted from a chapter in "Common Mistakes in Meta‐Analysis and How to Avoid Them." A free PDF of the book is available at https://www.Meta-Analysis.com/rsm. [ABSTRACT FROM AUTHOR]

A recent paper proposed an alternative weighting scheme when performing matching‐adjusted indirect comparisons. This alternative approach follows the conventional one in matching the covariate means across two studies but differs in that it maximizes the effective sample size when doing so. The appendix of this paper showed, assuming there is one covariate and negative weights are permitted, that the resulting weights are linear in the covariates. This explains how the alternative method achieves a larger effective sample size and results in a metric that quantifies the difficulty of matching on particular covariates. We explain how these key results generalize to the case where there are multiple covariates, giving rise to a new metric that can be used to quantify the impact of matching on multiple covariates. [ABSTRACT FROM AUTHOR]

Systematic reviews are vital instruments for researchers to understand broad trends in a field and synthesize evidence on the effectiveness of interventions in addressing specific issues. The quality of a systematic review depends critically on having comprehensively surveyed all relevant literature on the review topic. In addition to database searching, handsearching is an important supplementary technique that helps increase the likelihood of identifying all relevant studies in a literature search. Traditional handsearching requires reviewers to manually browse through a curated list of field‐specific journals and conference proceedings to find articles relevant to the review topic. This manual process is not only time‐consuming, laborious, costly, and error‐prone due to human fatigue, but it also lacks replicability due to its cumbersome manual nature. To address these issues, this paper presents a free and open‐source Python package and an accompanying web‐app, Paperfetcher, to automate the retrieval of article metadata for handsearching. With Paperfetcher's assistance, researchers can retrieve article metadata from designated journals within a specified time frame in just a few clicks. In addition to handsearching, it also incorporates a beta version of citation searching in both forward and backward directions. Paperfetcher has an easy‐to‐use interface, which allows researchers to download the metadata of retrieved studies as a list of DOIs or as an RIS file to facilitate seamless import into systematic review screening software. To the best of our knowledge, Paperfetcher is the first tool to automate handsearching with high usability and a multi‐disciplinary focus. [ABSTRACT FROM AUTHOR]

The product Bayes factor (PBF) synthesizes evidence for an informative hypothesis across heterogeneous replication studies. It can be used when fixed‐ or random effects meta‐analysis fall short. For example, when effect sizes are incomparable and cannot be pooled, or when studies diverge significantly in the populations, study designs, and measures used. PBF shines as a solution for small sample meta‐analyses, where the number of between‐study differences is often large relative to the number of studies, precluding the use of meta‐regression to account for these differences. Users should be mindful of the fact that the PBF answers a qualitatively different research question than other evidence synthesis methods. For example, whereas fixed‐effect meta‐analysis estimates the size of a population effect, the PBF quantifies to what extent an informative hypothesis is supported in all included studies. This tutorial paper showcases the user‐friendly PBF functionality within the bain R‐package. This new implementation of an existing method was validated using a simulation study, available in an Online Supplement. Results showed that PBF had a high overall accuracy, due to greater sensitivity and lower specificity, compared to random‐effects meta‐analysis, individual participant data meta‐analysis, and vote counting. Tutorials demonstrate applications of the method on meta‐analytic and individual participant data. The example datasets, based on published research, are included in bain so readers can reproduce the examples and apply the code to their own data. The PBF is a promising method for synthesizing evidence for informative hypotheses across conceptual replications that are not suitable for conventional meta‐analysis. [ABSTRACT FROM AUTHOR]

The application of network meta‐analysis is becoming increasingly widespread, and for a successful implementation, it requires that the direct comparison result and the indirect comparison result should be consistent. Because of this, a proper detection of inconsistency is often a key issue in network meta‐analysis as whether the results can be reliably used as a clinical guidance. Among the existing methods for detecting inconsistency, two commonly used models are the design‐by‐treatment interaction model and the side‐splitting models. While the original side‐splitting model was initially estimated using a Bayesian approach, in this context, we employ the frequentist approach. In this paper, we review these two types of models comprehensively as well as explore their relationship by treating the data structure of network meta‐analysis as missing data and parameterizing the potential complete data for each model. Through both analytical and numerical studies, we verify that the side‐splitting models are specific instances of the design‐by‐treatment interaction model, incorporating additional assumptions or under certain data structure. Moreover, the design‐by‐treatment interaction model exhibits robust performance across different data structures on inconsistency detection compared to the side‐splitting models. Finally, as a practical guidance for inconsistency detection, we recommend utilizing the design‐by‐treatment interaction model when there is a lack of information about the potential location of inconsistency. By contrast, the side‐splitting models can serve as a supplementary method especially when the number of studies in each design is small, enabling a comprehensive assessment of inconsistency from both global and local perspectives. [ABSTRACT FROM AUTHOR]

The Systematic Review Toolbox aims provide a web‐based catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. Identifying publications surrounding specific systematic review tools is currently challenging, leading to a high screening burden for few eligible records. We aimed to develop a search strategy that could be regularly and automatically run to identify eligible records for the SR Toolbox, thus reducing time on task and burden for those involved. We undertook a mapping exercise to identify the PubMed IDs of papers indexed within the SR Toolbox. We then used the Yale MeSH Analyser and Visualisation of Similarities (VOS) Viewer text‐mining software to identify the most commonly used MeSH terms and text words within the eligible records. These MeSH terms and text words were combined using Boolean Operators into a search strategy for Ovid MEDLINE. Prior to the mapping exercise and search strategy development, 81 software tools and 55 'Other' tools were included within the SR Toolbox. Since implementation of the search strategy, 146 tools have been added. There has been an increase in tools added to the toolbox since the search was developed and its corresponding auto‐alert in MEDLINE was originally set up. Developing a search strategy based on a mapping exercise is an effective way of identifying new tools to support the systematic review process. Further research could be conducted to help prioritise records for screening to reduce reviewer burden further and to adapt the strategy for disciplines beyond healthcare. [ABSTRACT FROM AUTHOR]

We present the R package and web app statcheck to automatically detect statistical reporting inconsistencies in primary studies and meta‐analyses. Previous research has shown a high prevalence of reported p‐values that are inconsistent ‐ meaning a re‐calculated p‐value, based on the reported test statistic and degrees of freedom, does not match the author‐reported p‐value. Such inconsistencies affect the reproducibility and evidential value of published findings. The tool statcheck can help researchers to identify statistical inconsistencies so that they may correct them. In this paper, we provide an overview of the prevalence and consequences of statistical reporting inconsistencies. We also discuss the tool statcheck in more detail and give an example of how it can be used in a meta‐analysis. We end with some recommendations concerning the use of statcheck in meta‐analyses and make a case for better reporting standards of statistical results. [ABSTRACT FROM AUTHOR]

We respond to discussant comments on our paper "Twenty years of network meta‐analysis: Continuing controversies and recent developments" (https://doi.org/10.1002/jrsm.1700) and raise some additional points for consideration, including: the way in which methodological guidance is generated; integration of the estimand framework with evidence synthesis; and implications of the European Joint Clinical Assessment. We ask: what properties are required of population adjustment methods to enable transparent and consistent decision‐making? We also ask why individual patient data is not routinely made available to re‐imbursement authorities and clinical guideline developers. [ABSTRACT FROM AUTHOR]

Recently, Ades and colleagues discussed the controversies and advancements in network meta‐analysis (NMA) over the past two decades, discussing its reliability, assumptions, novel approaches, and provided some useful recommendations for the conduction of NMAs. The present discussion paper builds on the insights by Ades and colleagues, providing a roadmap for NMA applications, advancements in software and tools, and approaches designed to facilitate the assessment and interpretation of NMA findings. It also discusses the impact of NMA across disciplines, particularly for policymakers and guideline developers. Despite 20 years of NMA history, challenges remain in understanding and assessing assumptions, communicating and interpreting findings, and applying common approaches like network meta‐regression and NMA involving non‐randomized studies in readily available software. NMA has proven particularly valuable in clinical decision‐making, which highlights the need for additional training and interdisciplinary collaboration of knowledge users, including patient engagement, to enhance its adoption and address real‐world problems. [ABSTRACT FROM AUTHOR]

The current paper responds to the commentary on the article (doi:10.1002/jrsm.1605). We discuss our perspectives on the missing data mechanisms and models assumed and used in our simulation study while acknowledging the inherent generalizability limitations of any (simulation) study. The plausibility of the exact missing data mechanism is challenging to definitively identify in any applied dataset. We describe and justify our assumed scenario in meta‐regression that we investigated. We also revisit the performance of the deletion method and how it is tied into the assumed missingness model. Lastly, we reiterate the importance of conducting sensitivity analyses assessing different ways of handling missing data given different assumptions and offer this study as a starting point for future research. [ABSTRACT FROM AUTHOR]

The approach to identifying evidence for inclusion in realist reviews differs from that used in 'traditional' systematic reviews. Guidance suggests that realist reviews should be inclusive of diverse data from a range of sources, gathered in iterative searching cycles. Saturation is prioritised over exhaustiveness. Supplementary techniques such as citation snowballing are emphasised as potentially important sources of evidence. This paper describes the processes used to identify evidence in a selection of realist reviews focused on primary health care settings and examines the origin and type of evidence selected for inclusion. Data from five realist reviews were extracted from (a) reviewers' reference management libraries and (b) records kept by review teams. Although all reviews focused on primary health care, they used data from a wide range of document types and research designs, drawing on learning from multiple perspectives and settings, and sourced the documents containing this data in a variety of ways. Systematic searching of academic databases played an important role, supplementary search techniques such as snowballing were used to identify a significant proportion of documents included in the reviews. Our analysis demonstrates the diverse data sources used within realist reviews and the need for flexible, responsive efforts to identify relevant documents. Reviewers and information specialists should devise approaches to data gathering that reflect the individual needs of realist review projects and report these transparently. [ABSTRACT FROM AUTHOR]

Overviews synthesising the results of multiple systematic reviews help inform evidence‐based clinical practice. In this first of two companion papers, we evaluate the bibliometrics of overviews, including their prevalence and factors affecting citation rates and journal impact factor (JIF). We searched MEDLINE, Epistemonikos and Cochrane Database of Systematic Reviews (CDSR). We included overviews that: (a) synthesised reviews, (b) conducted a systematic search, (c) had a methods section and (d) examined a healthcare intervention. Multivariable regression was conducted to determine the association between citation density, JIF and six predictor variables. We found 1218 overviews published from 2000 to 2020; the majority (73%) were published in the most recent 5‐year period. We extracted a selection of these overviews (n = 541; 44%) dated from 2000 to 2018. The 541 overviews were published in 307 journals; CDSR (8%), PLOS ONE (3%) and Sao Paulo Medical Journal (2%) were the most prevalent. The majority (70%) were published in journals with impact factors between 0.05 and 3.97. We found a mean citation count of 10 overviews per year, published in journals with a mean JIF of 4.4. In multivariable analysis, overviews with a high number of citations and JIFs had more authors, larger sample sizes, were open access and reported the funding source. An eightfold increase in the number of overviews was found between 2009 and 2020. We identified 332 overviews published in 2020, which is equivalent to one overview published per day. Overviews perform above average for the journals in which they publish. [ABSTRACT FROM AUTHOR]

Systematic reviews and maps are considered a reliable form of research evidence, but often neglect non-English-language literature, which can be a source of important evidence. To understand the barriers that might limit authors' ability or intent to find and include non-English-language literature, we assessed factors that may predict the inclusion of non-English-language literature in ecological systematic reviews and maps, as well as the review authors' perspectives. We assessed systematic reviews and maps published in "Environmental Evidence" (n = 72). We also surveyed authors from each paper (n = 32 responses), gathering information on the barriers to the inclusion of non-English language literature. 44% of the reviewed papers (32/72) excluded non-English literature from their searches and inclusions. Commonly cited reasons included constraints related to resources and time. Regression analysis revealed that reviews with larger author teams, authors from diverse countries, especially those with non-English primary languages, and teams with multilingual capabilities searched in a significantly greater number of non-English languages. Our survey exposed limited language diversity within the review teams and inadequate funding as the principal barriers to incorporating non-English language literature. To improve language inclusion and reduce bias in systematic reviews and maps, our study suggests increasing language diversity within review teams. Combining machine translation with language skills can alleviate the financial and resource burdens of translation. Funding applications could also include translation costs. Additionally, establishing language exchange systems would enable access to information in more languages. Further studies investigating language inclusion in other journals would strengthen these conclusions.

Background Rapid reviews are increasingly used to replace/complement systematic reviews to support evidence-based decision-making. Little is known about how this expedited process affects results. Objectives To assess differences between rapid and systematic review approaches for a case study of test accuracy of succinylacetone for detecting tyrosinemia type 1. Methods Two reviewers conducted an 'enhanced' rapid review then a systematic review. The enhanced rapid review involved narrower searches, a single reviewer checking 20% of titles/abstracts and data extraction, and quality assessment using an unadjusted QUADAS-2. Two reviewers performed the systematic review with a tailored QUADAS-2. Post hoc analysis examined rapid reviewing with a single reviewer (basic rapid review). Results Ten papers were included. Basic rapid reviews would have missed 1 or 4 of these (dependent on which reviewer). Enhanced rapid and systematic reviews identified all 10 papers; one paper was only identified in the rapid review through reference checking. Two thousand one hundred seventy-six fewer title/abstracts and 129 fewer full texts were screened during the enhanced rapid review than the systematic review. The unadjusted QUADAS-2 generated more 'unclear' ratings than the adjusted QUADAS-2 [29/70 (41.4%) versus 16/70 (22.9%)], and fewer 'high' ratings [22/70 (31.4%) versus 42/70 (60.0%)]. Basic rapid reviews contained important inaccuracies in data extraction, which were detected by a second reviewer in the enhanced rapid and systematic reviews. Conclusions Enhanced rapid reviews with 20% checking by a second reviewer may be an appropriate tool for policymakers to expeditiously assess evidence. Basic rapid reviews (single reviewer) have higher risks of important inaccuracies and omissions. [ABSTRACT FROM AUTHOR]

Many statistical methods (estimators) are available for estimating the consensus value (or average effect) and heterogeneity variance in interlaboratory studies or meta‐analyses. These estimators are all valid because they are developed from or supported by certain statistical principles. However, no estimator can be perfect and must have error or uncertainty (known as estimator uncertainty). For a given dataset, the consensus value and heterogeneity variance given by different estimators can often differ significantly. Consequently, the choice of different estimators can affect the conclusion of an interlaboratory study or meta‐analysis. However, there is no universally accepted metric for determining which estimator is optimal among a set of candidate estimators. Instead of selecting and using a single estimator, this paper proposes an estimator‐averaging approach to combine a set of individual estimators. The final averaged estimator is a linear combination of individual estimators, which accounts for three sources of uncertainties including the estimator uncertainty. Monte Carlo simulations were performed to examine the long‐run performance of four individual estimators and the proposed averaged estimators. A case study: the determination of the Newtonian constant of gravitation is presented, where 10 individual estimators (eight frequentist weighted average methods and two Bayesian methods) are combined using the proposed estimator‐averaging approach. [ABSTRACT FROM AUTHOR]

Evidence synthesis findings depend on the assumption that the included studies follow good clinical practice and results are not fabricated or false. Studies which are problematic due to scientific misconduct, poor research practice, or honest error may distort evidence synthesis findings. Authors of evidence synthesis need transparent mechanisms to identify and manage problematic studies to avoid misleading findings. As evidence synthesis authors of the Cochrane COVID‐19 review on ivermectin, we identified many problematic studies in terms of research integrity and regulatory compliance. Through iterative discussion, we developed a research integrity assessment (RIA) tool for randomized controlled trials for the update of this Cochrane review. In this paper, we explain the rationale and application of the RIA tool in this case study. RIA assesses six study criteria: study retraction, prospective trial registration, adequate ethics approval, author group, plausibility of methods (e.g., randomization), and plausibility of study results. RIA was used in the Cochrane review as part of the eligibility check during screening of potentially eligible studies. Problematic studies were excluded and studies with open questions were held in awaiting classification until clarified. RIA decisions were made independently by two authors and reported transparently. Using the RIA tool resulted in the exclusion of >40% of studies in the first update of the review. RIA is a complementary tool prior to assessing "Risk of Bias" aiming to establish the integrity and authenticity of studies. RIA provides a platform for urgent development of a standard approach to identifying and managing problematic studies. [ABSTRACT FROM AUTHOR]

Having surveyed the history and methods of meta‐regression in a previous paper, in this paper, we review which and how meta‐regression methods are applied in recent research syntheses. To do so, we reviewed studies published in 2016 across four leading research synthesis journals: Psychological Bulletin, the Journal of Applied Psychology, Review of Educational Research, and the Cochrane Library. We find that the best practices defined in the previous review are rarely carried out in practice. In light of the identified discrepancies, we consider how to move forward, first by identifying areas where further methods development is needed to address persistent problems in the field and second by discussing how to more effectively disseminate points of methodological consensus. [ABSTRACT FROM AUTHOR]

At the beginning of the development of meta‐analysis, understanding the role of moderators was given the highest priority, with meta‐regression provided as a method for achieving this goal. Yet in current practice, meta‐regression is not as commonly used as anticipated. This paper seeks to understand this mismatch by reviewing the history of meta‐regression methods over the past 40 years. We divide this time span into four periods and examine three types of methodological developments within each period: technical, conceptual, and practical. Our focus is broad and includes development of methods in the fields of education, psychology, and medicine. We conclude the paper with a discussion of five consensus points, as well as open questions and areas of research for the future. [ABSTRACT FROM AUTHOR]

Meta‐analysis and structural equation modeling (SEM) are 2 of the most prominent statistical techniques employed in the behavioral, medical, and social sciences. They each have their own well‐established research communities, terminologies, statistical models, software packages, and journals (Research Synthesis Methods and Structural Equation Modeling: A Multidisciplinary Journal). In this paper, I will provide some personal reflections on combining meta‐analysis and SEM in the forms of meta‐analytic SEM and SEM‐based meta‐analysis. The critical contributions of Becker (1992), Shadish (1992), and Viswesvaran and Ones (1995) in the early development of meta‐analytic SEM are highlighted. Another goal of the paper is to illustrate how meta‐analysis can be extended and integrated with other techniques to address new research questions such as the analysis of Big Data. I hope that this paper may stimulate more research development in the area of combining meta‐analysis and SEM. [ABSTRACT FROM AUTHOR]

In response to the evolving needs of knowledge synthesis, this manuscript introduces the concept of narrative reanalysis, a method that refines data from initial reviews, such as systematic and reviews, to focus on specific sub-phenomena. Unlike traditional narrative reviews, which lack the methodological rigor of systematic reviews and are broader in scope, our methodological framework for narrative reanalysis applies a structured, systematic framework to the interpretation of existing data. This approach enables a focused investigation of nuanced topics within a broader dataset, enhancing understanding and generating new insights. We detail a five-stage methodological framework that guides the narrative reanalysis process: (1) retrieval of an initial review, (2) identification and justification of a sub-phenomenon, (3) expanded search, selection, and extraction of data, (4) reanalyzing the sub-phenomenon, and (5) writing the report. The proposed framework aims to standardize narrative reanalysis, advocating for its use in academic and research settings to foster more rigorous and insightful literature reviews. This approach bridges the methodological gap between narrative and systematic reviews, offering a valuable tool for researchers to explore detailed aspects of broader topics without the extensive resources required for systematic reviews., (© 2024 The Author(s). Research Synthesis Methods published by John Wiley & Sons Ltd.)

Increasing concerns about the trustworthiness of research have prompted calls to scrutinise studies' Individual Participant Data (IPD), but guidance on how to do this was lacking. To address this, we developed the IPD Integrity Tool to screen randomised controlled trials (RCTs) for integrity issues. Development of the tool involved a literature review, consultation with an expert advisory group, piloting on two IPD meta-analyses (including 73 trials with IPD), preliminary validation on 13 datasets with and without known integrity issues, and evaluation to inform iterative refinements. The IPD Integrity Tool comprises 31 items (13 study-level, 18 IPD-specific). IPD-specific items are automated where possible, and are grouped into eight domains, including unusual data patterns, baseline characteristics, correlations, date violations, patterns of allocation, internal and external inconsistencies, and plausibility of data. Users rate each item as having either no issues, some/minor issue(s), or many/major issue(s) according to decision rules, and justification for each rating is recorded. Overall, the tool guides decision-making by determining whether a trial has no concerns, some concerns requiring further information, or major concerns warranting exclusion from evidence synthesis or publication. In our preliminary validation checks, the tool accurately identified all five studies with known integrity issues. The IPD Integrity Tool enables users to assess the integrity of RCTs via examination of IPD. The tool may be applied by evidence synthesists, editors and others to determine whether an RCT should be considered sufficiently trustworthy to contribute to the evidence base that informs policy and practice., (© 2024 The Author(s). Research Synthesis Methods published by John Wiley & Sons Ltd.)

Existing systems for automating the assessment of risk-of-bias (RoB) in medical studies are supervised approaches that require substantial training data to work well. However, recent revisions to RoB guidelines have resulted in a scarcity of available training data. In this study, we investigate the effectiveness of generative large language models (LLMs) for assessing RoB. Their application requires little or no training data and, if successful, could serve as a valuable tool to assist human experts during the construction of systematic reviews. Following Cochrane's latest guidelines (RoB2) designed for human reviewers, we prepare instructions that are fed as input to LLMs, which then infer the risk associated with a trial publication. We distinguish between two modelling tasks: directly predicting RoB2 from text; and employing decomposition, in which a RoB2 decision is made after the LLM responds to a series of signalling questions. We curate new testing data sets and evaluate the performance of four general- and medical-domain LLMs. The results fall short of expectations, with LLMs seldom surpassing trivial baselines. On the direct RoB2 prediction test set (n = 5993), LLMs perform akin to the baselines (F1: 0.1-0.2). In the decomposition task setup (n = 28,150), similar F1 scores are observed. Our additional comparative evaluation on RoB1 data also reveals results substantially below those of a supervised system. This testifies to the difficulty of solving this task based on (complex) instructions alone. Using LLMs as an assisting technology for assessing RoB2 thus currently seems beyond their reach., (© 2024 The Author(s). Research Synthesis Methods published by John Wiley & Sons Ltd.)

Literature screening is the process of identifying all relevant records from a pool of candidate paper records in systematic review, meta-analysis, and other research synthesis tasks. This process is time consuming, expensive, and prone to human error. Screening prioritization methods attempt to help reviewers identify most relevant records while only screening a proportion of candidate records with high priority. In previous studies, screening prioritization is often referred to as automatic literature screening or automatic literature identification. Numerous screening prioritization methods have been proposed in recent years. However, there is a lack of screening prioritization methods with reliable performance. Our objective is to develop a screening prioritization algorithm with reliable performance for practical use, for example, an algorithm that guarantees an 80% chance of identifying at least 80% of the relevant records. Based on a target-based method proposed in Cormack and Grossman, we propose a screening prioritization algorithm using sampling with replacement. The algorithm is a wrapper algorithm that can work with any current screening prioritization algorithm to guarantee the performance. We prove, with mathematics and probability theory, that the algorithm guarantees the performance. We also run numeric experiments to test the performance of our algorithm when applied in practice. The numeric experiment results show this algorithm achieve reliable performance under different circumstances. The proposed screening prioritization algorithm can be reliably used in real world research synthesis tasks. [ABSTRACT FROM AUTHOR]

Qualitative comparative analysis (QCA) is a hybrid method designed to bridge the gap between qualitative and quantitative research in a case-sensitive approach that considers each case holistically as a complex configuration of conditions and outcomes. QCA allows for multiple conjunctural causation, implying that it is often a combination of conditions that produces an outcome, that multiple pathways may lead to the same outcome, and that in different contexts, the same condition may have a different impact on the outcome. This approach to complexity allows QCA to provide a practical understanding for complex, real-world situations, and the context of implementing interventions. There are guides for conducting QCA in primary research and quantitative systematic reviews yet, to our knowledge, no guidance for conducting QCA in systematic mixed studies reviews (SMSRs). Thus, the specific objectives of this paper are to (1) describe a step-by-step approach for novice researchers for using QCA to integrate qualitative and quantitative evidence, including guidance on how to use software; (2) highlight specific challenges; (3) propose potential solutions from a worked example; and (4) provide recommendations for reporting. [ABSTRACT FROM AUTHOR]

Effect modification (EM) may cause bias in network meta‐analysis (NMA). Existing population adjustment NMA methods use individual patient data to adjust for EM but disregard available subgroup information from aggregated data in the evidence network. Additionally, these methods often rely on the shared effect modification (SEM) assumption. In this paper, we propose Network Meta‐Interpolation (NMI): a method using subgroup analyses to adjust for EM that does not assume SEM. NMI balances effect modifiers across studies by turning treatment effect (TE) estimates at the subgroup‐ and study level into TE and standard errors at EM values common to all studies. In an extensive simulation study, we simulate two evidence networks consisting of four treatments, and assess the impact of departure from the SEM assumption, variable EM correlation across trials, trial sample size and network size. NMI was compared to standard NMA, network meta‐regression (NMR) and Multilevel NMR (ML‐NMR) in terms of estimation accuracy and credible interval (CrI) coverage. In the base case non‐SEM dataset, NMI achieved the highest estimation accuracy with root mean squared error (RMSE) of 0.228, followed by standard NMA (0.241), ML‐NMR (0.447) and NMR (0.541). In the SEM dataset, NMI was again the most accurate method with RMSE of 0.222, followed by ML‐NMR (0.255). CrI coverage followed a similar pattern. NMI's dominance in terms of estimation accuracy and CrI coverage appeared to be consistent across all scenarios. NMI represents an effective option for NMA in the presence of study imbalance and available subgroup data. [ABSTRACT FROM AUTHOR]

Context Meta-analyses are regularly used to inform healthcare decisions. Concerns have been expressed about the quality of meta-analyses and, in particular, about those supported by the pharmaceutical industry. Objective The objective of this study is to compare the quality of pharmaceutical-industry-supported meta-analyses with academic meta-analyses and of meta-analyses published before and after companies started to disclose their data. Data Sources We identified industry-supported meta-analyses by searching the Scopus bibliographic database, using author affiliations. We matched each industry-supported meta-analysis with an academic meta-analysis using high-level MeSH terms in PubMed. Study Selection We included meta-analyses of randomized trials assessing the efficacy or safety of any pharmaceutical intervention in humans, published in 2002-2004 or 2008-2009. Cochrane reviews were excluded. Two individuals independently selected papers, with discrepancies resolved by two further individuals. Assessment We developed and piloted a quality-assessment tool, consisting of 43 questions in four domains, with a key summary question covering each domain. Two individuals independently assessed each meta-analysis. Results We examined 126 meta-analysis publications in 63 matched pairs. The average quality was low, with fewer than 50% adequate in three of the four domains. Industry-supported meta-analyses less often demonstrated adequate methods for locating studies and assessing their quality (odds ratio 0.44, 95% confidence interval 0.21 to 0.92), for analysing the included studies (0.52, 0.25 to 1.06), for undertaking meta-analyses (0.82, 0.40 to 1.68) and in reaching sound conclusions (0.62, 0.30 to 1.28). Quality generally improved over time, particularly for some aspects of industry reports. Conclusions Academic meta-analysis papers are generally of higher quality than industry-supported ones. This is largely due to less detailed reporting in industry-supported meta-analyses and a tendency for them to take the included studies at face value, probably arising from the implicit assumption that these studies already have high methodological standards to meet licensing requirements. The improved quality over time does not appear to be due to the use of data disclosed by industry. The main limitations of this study are the small sample of papers and the subjective nature of some of the assessment processes. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

A recent paper proposed an alternative weighting scheme when performing matching-adjusted indirect comparisons. This alternative approach follows the conventional one in matching the covariate means across two studies but differs in that it maximizes the effective sample size when doing so. The appendix of this paper showed, assuming there is one covariate and negative weights are permitted, that the resulting weights are linear in the covariates. This explains how the alternative method achieves a larger effective sample size and results in a metric that quantifies the difficulty of matching on particular covariates. We explain how these key results generalize to the case where there are multiple covariates, giving rise to a new metric that can be used to quantify the impact of matching on multiple covariates.

In any meta-analysis, it is critically important to report the dispersion in effects as well as the mean effect. If an intervention has a moderate clinical impact "on average" we also need to know if the impact is moderate for all relevant populations, or if it varies from trivial in some to major in others. Or indeed, if the intervention is beneficial in some cases but harmful in others. Researchers typically report a series of statistics such as the Q-value, the p-value, and I[superscript 2], which are intended to address this issue. Often, they use these statistics to classify the heterogeneity as being low, moderate, or high and then use these classifications when considering the potential utility of the intervention. While this practice is ubiquitous, it is nevertheless incorrect. The statistics mentioned above do not actually tell us how much the effect size varies. Classifications of heterogeneity based on these statistics are uninformative at best, and often misleading. My goal in this paper is to explain what these statistics do tell us, and that none of them tells us how much the effect size varies. Then I will introduce the prediction interval, the statistic that does tell us how much the effect size varies, and that addresses the question we have in mind when we ask about heterogeneity. This paper is adapted from a chapter in "Common Mistakes in Meta-Analysis and How to Avoid Them." A free PDF of the book is available at https://www.Meta-Analysis.com/rsm.

The amount of grey literature and 'softer' intelligence from social media or websites is vast. Given the long lead-times of producing high-quality peer-reviewed health information, this is causing a demand for new ways to provide prompt input for secondary research. To our knowledge, this is the first review of automated data extraction methods or tools for health-related grey literature and soft data, with a focus on (semi)automating horizon scans, health technology assessments (HTA), evidence maps, or other literature reviews. We searched six databases to cover both health- and computer-science literature. After deduplication, 10% of the search results were screened by two reviewers, the remainder was single-screened up to an estimated 95% sensitivity; screening was stopped early after screening an additional 1000 results with no new includes. All full texts were retrieved, screened, and extracted by a single reviewer and 10% were checked in duplicate. We included 84 papers covering automation for health-related social media, internet fora, news, patents, government agencies and charities, or trial registers. From each paper, we extracted data about important functionalities for users of the tool or method; information about the level of support and reliability; and about practical challenges and research gaps. Poor availability of code, data, and usable tools leads to low transparency regarding performance and duplication of work. Financial implications, scalability, integration into downstream workflows, and meaningful evaluations should be carefully planned before starting to develop a tool, given the vast amounts of data and opportunities those tools offer to expedite research.

Selective publication and reporting in individual papers compromise the scientific record, but are meta‐analyses as compromised as their constituent studies? We systematically sampled 63 meta‐analyses (each comprising at least 40 studies) in PLoS One, top medical journals, top psychology journals, and Metalab, an online, open‐data database of developmental psychology meta‐analyses. We empirically estimated publication bias in each, including only the peer‐reviewed studies. Across all meta‐analyses, we estimated that "statistically significant" results in the expected direction were only 1.17 times more likely to be published than "nonsignificant" results or those in the unexpected direction (95% CI: [0.93, 1.47]), with a confidence interval substantially overlapping the null. Comparable estimates were 0.83 for meta‐analyses in PLoS One, 1.02 for top medical journals, 1.54 for top psychology journals, and 4.70 for Metalab. The severity of publication bias did differ across individual meta‐analyses; in a small minority (10%; 95% CI: [2%, 21%]), publication bias appeared to favor "significant" results in the expected direction by more than threefold. We estimated that for 89% of meta‐analyses, the amount of publication bias that would be required to attenuate the point estimate to the null exceeded the amount of publication bias estimated to be actually present in the vast majority of meta‐analyses from the relevant scientific discipline (exceeding the 95th percentile of publication bias). Study‐level measures ("statistical significance" with a point estimate in the expected direction and point estimate size) did not indicate more publication bias in higher‐tier versus lower‐tier journals, nor in the earliest studies published on a topic versus later studies. Overall, we conclude that the mere act of performing a meta‐analysis with a large number of studies (at least 40) and that includes non‐headline results may largely mitigate publication bias in meta‐analyses, suggesting optimism about the validity of meta‐analytic results. [ABSTRACT FROM AUTHOR]

Evidence‐based decision‐making is predicated on the ability of users to find and comprehend results from systematic review. Evidence producers have an obligation to support evidence users in this process. The Agency for Healthcare Research and Quality (AHRQ) Evidence‐based Practice Center (EPC) program—a producer of rigorous and comprehensive systematic reviews for two decades—has set a gold standard for reliability in health evidence reviews in the United States. It has recently begun a program of active support for evidence dissemination and uptake beyond mere publication of lengthy reports. This Brief Methods Note critiques the current paper‐based format for systematic reviews and describes the development of a next generation (NxGen) AHRQ EPC Effective Health Care website. This redesigned platform will allow end‐users of all types to find and share the evidence they need through data visualizations and other interactive displays. Several design principles guided the development of NxGen to make systematic review findings more accessible, customizable, adaptable, interactive, and shareable. NxGen will include visualizations for report results that are expressed as meta‐analyses as well as those with narrative syntheses, through forest or bubble plots, respectively. Visual and interactive evidence heat maps are also planned. The NxGen version of the Effective Health Care website is planned to go live in the latter half of 2020 or early 2021. [ABSTRACT FROM AUTHOR]

Since the early 1990s the number of systematic reviews (SR) of animal studies has steadily increased. There is, however, little guidance on when and how to conduct a meta‐analysis of human‐health‐related animal studies. To gain insight about the methods that are currently used we created an overview of the key characteristics of published meta‐analyses of animal studies, with a focus on the choice of effect size measures. An additional goal was to learn about the rationale behind the meta‐analysis methods used by the review authors. We show that important details of the meta‐analyses are not fully described, only a fraction of all human‐health‐related meta‐analyses provided rationales for their decision to use specific effect size measures. In addition, our data may suggest that authors make post‐hoc decisions to switch to another effect size measure during the course of their meta‐analysis, and possibly search for significant effects. Based on analyses in this paper we recommend that review teams: 1) publish a review protocol before starting the conduct of a SR, prespecifying all methodological details (providing special attention to the planned meta‐analysis including the effect size measure and the rational behind choosing a specific effect size, prespecifying subgroups and restricting the number of subgroup analyses), 2) always use the Preferred Reporting Items for Systematic Review and Meta‐Analyses (PRISMA) checklist to report your SR of animal studies, and 3) use the random effects model (REM) in human‐health‐related meta‐analysis of animal studies, unless the assumptions for using the fixed effect model (FEM) are all met. [ABSTRACT FROM AUTHOR]