Search

The application of network meta-analysis is becoming increasingly widespread, and for a successful implementation, it requires that the direct comparison result and the indirect comparison result should be consistent. Because of this, a proper detection of inconsistency is often a key issue in network meta-analysis as whether the results can be reliably used as a clinical guidance. Among the existing methods for detecting inconsistency, two commonly used models are the design-by-treatment interaction model and the side-splitting models. While the original side-splitting model was initially estimated using a Bayesian approach, in this context, we employ the frequentist approach. In this paper, we review these two types of models comprehensively as well as explore their relationship by treating the data structure of network meta-analysis as missing data and parameterizing the potential complete data for each model. Through both analytical and numerical studies, we verify that the side-splitting models are specific instances of the design-by-treatment interaction model, incorporating additional assumptions or under certain data structure. Moreover, the design-by-treatment interaction model exhibits robust performance across different data structures on inconsistency detection compared to the side-splitting models. Finally, as a practical guidance for inconsistency detection, we recommend utilizing the design-by-treatment interaction model when there is a lack of information about the potential location of inconsistency. By contrast, the side-splitting models can serve as a supplementary method especially when the number of studies in each design is small, enabling a comprehensive assessment of inconsistency from both global and local perspectives.

A diversity of approaches for critically appraising qualitative and quantitative evidence exist and emphasize different aspects. These approaches lack clear processes to facilitate rating the overall quality of the evidence for aggregated findings that combine qualitative and quantitative evidence. We draw on a meta-aggregation of implementation and process evaluations to illustrate a method for critically appraising empirical findings generated from qualitative and quantitative studies. This method includes a rubric for standardizing assessments of the overall quality of evidence in an evidence synthesis or mixed-method systematic review. The method first assesses the credibility of each finding extracted from a study. These individual assessments then feed into an overall score for any synthesized finding generated from the meta-aggregation. We argue that this approach provides a balanced and inclusive method of critical appraisal by first assessing individual findings, rather than studies, using flexible criteria applicable to a range of primary study methods to derive an overall assessment of synthesized findings., (© 2023 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.)

Publication bias distorts the available empirical evidence and misinforms policymaking. Evidence of publication bias is mounting in virtually all fields of empirical research. This paper proposes the endogenous kink (EK) meta‐regression model as a novel method of publication bias correction. The EK method fits a piecewise linear meta‐regression of the primary estimates on their standard errors, with a kink at the cutoff value of the standard error below which publication selection is unlikely. We provide a simple method of endogenously determining this cutoff value as a function of a first‐stage estimate of the true effect and an assumed threshold of statistical significance. Our Monte Carlo simulations show that EK is less biased and more efficient than other related regression‐based methods of publication bias correction in a variety of research conditions. [ABSTRACT FROM AUTHOR]

Abstract screening is one important aspect of conducting a high‐quality and comprehensive systematic review and meta‐analysis. Abstract screening allows the review team to conduct the tedious but vital first step to synthesize the extant literature: winnowing down the overwhelming amalgamation of citations discovered through research databases to the citations that should be "full‐text" screened and eventually included in the review. Although it is a critical process, few guidelines have been put forth since the publications of seminal systematic review textbooks. The purpose of this paper, therefore, is to provide a practical set of best practice guidelines to help future review teams and managers. Each of the 10 proposed guidelines is explained using real‐world examples or illustrations from applications. We also delineate recent experiences where a team of abstract screeners double‐screened 14 923 abstracts in 89 days. [ABSTRACT FROM AUTHOR]

Copyright of Research Synthesis Methods is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Background: Both PubMed and Ovid MEDLINE contain records from the MEDLINE database. However, there are subtle differences in content, functionality, and search syntax between the two. There are many instances in which researchers may wish to search both interfaces, such as when conducting supplementary searching for a systematic review to retrieve a unique content from PubMed or when using a previously published search strategy from a different interface, but little guidance on how to best conduct these searches. The aim of this project is to describe differences in search functionality between Ovid MEDLINE and PubMed, provide guidance for converting search strategies between the two, and develop an easy‐to‐use, freely available web‐based tool to automate search syntax translations. Case presentation: In this paper, we present a custom‐built freely available online tool, Medline Transpose, to streamline the process of converting search strategies between Ovid MEDLINE and PubMed. With this tool, users can paste a strategy formatted for one interface into the search box and immediately retrieve an output formatted for use in the other interface, with recommendations for changes that users can make to the strategy where an exact translation does not exist. Conclusion: This novel approach has the potential to reduce time and errors that database users spend translating search strategies. [ABSTRACT FROM AUTHOR]

Meta‐analyses are an important tool within systematic reviews to estimate the overall effect size and its confidence interval for an outcome of interest. If heterogeneity between the results of the relevant studies is anticipated, then a random‐effects model is often preferred for analysis. In this model, a prediction interval for the true effect in a new study also provides additional useful information. However, the DerSimonian and Laird method—frequently used as the default method for meta‐analyses with random effects—has been long challenged due to its unfavorable statistical properties. Several alternative methods have been proposed that may have better statistical properties in specific scenarios. In this paper, we aim to provide a comprehensive overview of available methods for calculating point estimates, confidence intervals, and prediction intervals for the overall effect size under the random‐effects model. We indicate whether some methods are preferable than others by considering the results of comparative simulation and real‐life data studies. [ABSTRACT FROM AUTHOR]

Studies combined in a meta‐analysis often have differences in their design and conduct that can lead to heterogeneous results. A random‐effects model accounts for these differences in the underlying study effects, which includes a heterogeneity variance parameter. The DerSimonian‐Laird method is often used to estimate the heterogeneity variance, but simulation studies have found the method can be biased and other methods are available. This paper compares the properties of nine different heterogeneity variance estimators using simulated meta‐analysis data. Simulated scenarios include studies of equal size and of moderate and large differences in size. Results confirm that the DerSimonian‐Laird estimator is negatively biased in scenarios with small studies and in scenarios with a rare binary outcome. Results also show the Paule‐Mandel method has considerable positive bias in meta‐analyses with large differences in study size. We recommend the method of restricted maximum likelihood (REML) to estimate the heterogeneity variance over other methods. However, considering that meta‐analyses of health studies typically contain few studies, the heterogeneity variance estimate should not be used as a reliable gauge for the extent of heterogeneity in a meta‐analysis. The estimated summary effect of the meta‐analysis and its confidence interval derived from the Hartung‐Knapp‐Sidik‐Jonkman method are more robust to changes in the heterogeneity variance estimate and show minimal deviation from the nominal coverage of 95% under most of our simulated scenarios. [ABSTRACT FROM AUTHOR]

This paper proposes a new method to evaluate informative hypotheses for meta-analysis of Cronbach's coefficient alpha using a Bayesian approach. The coefficient alpha is one of the most widely used reliability indices. In meta-analyses of reliability, researchers typically form specific informative hypotheses beforehand, such as 'alpha of this test is greater than 0.8' or 'alpha of one form of a test is greater than the others.' The proposed method enables direct evaluation of these informative hypotheses. To this end, a Bayes factor is calculated to evaluate the informative hypothesis against its complement. It allows researchers to summarize the evidence provided by previous studies in favor of their informative hypothesis. The proposed approach can be seen as a natural extension of the Bayesian meta-analysis of coefficient alpha recently proposed in this journal (Brannick and Zhang, ). The proposed method is illustrated through two meta-analyses of real data that evaluate different kinds of informative hypotheses on superpopulation: one is that alpha of a particular test is above the criterion value, and the other is that alphas among different test versions have ordered relationships. Informative hypotheses are supported from the data in both cases, suggesting that the proposed approach is promising for application. Copyright © 2015 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

The Handbook of Meta-Analysis, edited by three outstanding statisticians with contributions by the editors and many other meta-analysis experts, provides a detailed overview of the current state of meta-analysis on more than 500 pages.1 It consists of an introduction, three chapters on general themes and methods in meta-analysis, followed by 19 more special chapters, covering exact likelihood methods, Bayesian methods, meta-regression, multivariate meta-analysis including network meta-analysis, and individual participant data. Interestingly, in their recent RSM paper two authors of this chapter discuss all these methods.7 Chapter 19 (Meta-analysis of diagnostic test accuracy studies, MA-DT) is restricted to a binary diagnostic test (positive or negative) and a binary disease status (present or absent). Unfortunately, re-running the available R and Stata code was not possible for seven chapters: the data set (with aggregate data, not individual participant data) was missing for three chapters and running the code resulted in an error message for four chapters. [Extracted from the article]

There is ongoing interest in including grey literature in systematic reviews. Including grey literature can broaden the scope to more relevant studies, thereby providing a more complete view of available evidence. Searching for grey literature can be challenging despite greater access through the Internet, search engines and online bibliographic databases. There are a number of publications that list sources for finding grey literature in systematic reviews. However, there is scant information about how searches for grey literature are executed and how it is included in the review process. This level of detail is important to ensure that reviews follow explicit methodology to be systematic, transparent and reproducible. The purpose of this paper is to provide a detailed account of one systematic review team's experience in searching for grey literature and including it throughout the review. We provide a brief overview of grey literature before describing our search and review approach. We also discuss the benefits and challenges of including grey literature in our systematic review, as well as the strengths and limitations to our approach. Detailed information about incorporating grey literature in reviews is important in advancing methodology as review teams adapt and build upon the approaches described. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Driven by necessity in our own complex review, we developed alternative systematic ways of identifying relevant evidence where the key concepts are generally not focal to the primary studies' aims and are found across multiple disciplines-that is, hard-to-detect evidence. Specifically, we sought to identify evidence on community engagement in public health interventions that aim to reduce health inequalities. Our initial search strategy used text mining to identify synonyms for the concept 'community engagement'. We conducted a systematic search for reviews on public health interventions, supplemented by searches of trials databases. We then used information in the reviews' evidence tables to gather more information about the included studies than was evident in the primary studies' own titles or abstracts. We identified 319 primary studies cited in reviews after full-text screening. In this paper, we retrospectively reflect on the challenges and benefits of the approach taken. We estimate that more than a quarter of the studies that were identified would have been missed by typical searching and screening methods. This identification strategy was highly effective and could be useful for reviews of broad research questions, or where the key concepts are unlikely to be the main focus of primary research. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

When the Medical Library Association identified questions critical for the future of the profession, it assigned groups to use systematic reviews to find the answers to these questions. Group 6, whose question was on emerging technologies, recognized early on that the systematic review process would not work well for this question, which looks forward to predict future trends, whereas the systematic review process looks back in time. We searched for new methodologies that were more appropriate to our question, developing a process that combined systematic review, text mining, and visualization techniques. We then discovered tech mining, which is very similar to the process we had created. In this paper, we describe our research design and compare tech mining and systematic review methodologies. There are similarities and differences in each process: Both use a defined research question, deliberate database selection, careful and iterative search strategy development, broad data collection, and thoughtful data analysis. However, the focus of the research differs significantly, with systematic reviews looking to the past and tech mining mainly to the future. Our comparison demonstrates that each process can be enhanced from a purposeful consideration of the procedures of the other. Tech mining would benefit from the inclusion of a librarian on their research team and a greater attention to standards and collaboration in the research project. Systematic reviews would gain from the use of tech mining tools to enrich their data analysis and corporate management communication techniques to promote the adoption of their findings. [ABSTRACT FROM AUTHOR]

Systematic reviews often encounter primary studies that report multiple effect sizes based on data from the same participants. These have the potential to introduce statistical dependency into the meta‐analytic data set. In this paper, we provide a tutorial on dealing with effect size multiplicity within studies in the context of meta‐analyses of intervention and association studies, recommending a three‐step approach. The first step is to define the research question and consider the extent to which it mainly reflects interest in mean effect sizes (which we term a convergent approach) or an interest in exploring heterogeneity (which we term a divergent approach). A second step is to identify the types of multiplicities that appear in the initial database of effect sizes relevant to the research question, and we propose a categorization scheme to differentiate them. The third step is to select a strategy for dealing with each type of multiplicity. The researcher can choose between a reductionist meta‐analytic approach, which is characterized by inclusion of a single effect size per study, and an integrative approach, characterized by inclusion of multiple effect sizes per study. We present an overview of available analysis strategies for dealing with effect size multiplicity within studies and provide recommendations intended to help researchers decide which strategy might be preferable in particular situations. Last, we offer caveats and cautions about addressing the challenges multiplicity poses for systematic reviews and meta‐analyses. [ABSTRACT FROM AUTHOR]

Small‐study effects are a common threat in systematic reviews and may indicate publication bias. Their existence is often verified by visual inspection of the funnel plot. Formal tests to assess the presence of funnel plot asymmetry typically estimate the association between the reported effect size and their standard error, the total sample size, or the inverse of the total sample size. In this paper, we demonstrate that the application of these tests may be less appropriate in meta‐analysis of survival data, where censoring influences statistical significance of the hazard ratio. We subsequently propose 2 new tests that are based on the total number of observed events and adopt a multiplicative variance component. We compare the performance of the various funnel plot asymmetry tests in an extensive simulation study where we varied the true hazard ratio (0.5 to 1), the number of published trials (N=10 to 100), the degree of censoring within trials (0% to 90%), and the mechanism leading to participant dropout (noninformative versus informative). Results demonstrate that previous well‐known tests for detecting funnel plot asymmetry suffer from low power or excessive type‐I error rates in meta‐analysis of survival data, particularly when trials are affected by participant dropout. Because our novel test (adopting estimates of the asymptotic precision as study weights) yields reasonable power and maintains appropriate type‐I error rates, we recommend its use to evaluate funnel plot asymmetry in meta‐analysis of survival data. The use of funnel plot asymmetry tests should, however, be avoided when there are few trials available for any meta‐analysis. [ABSTRACT FROM AUTHOR]

Multivariate meta‐analysis, which jointly analyzes multiple and possibly correlated outcomes in a single analysis, is becoming increasingly popular in recent years. An attractive feature of the multivariate meta‐analysis is its ability to account for the dependence between multiple estimates from the same study. However, standard inference procedures for multivariate meta‐analysis require the knowledge of within‐study correlations, which are usually unavailable. This limits standard inference approaches in practice. Riley et al proposed a working model and an overall synthesis correlation parameter to account for the marginal correlation between outcomes, where the only data needed are those required for a separate univariate random‐effects meta‐analysis. As within‐study correlations are not required, the Riley method is applicable to a wide variety of evidence synthesis situations. However, the standard variance estimator of the Riley method is not entirely correct under many important settings. As a consequence, the coverage of a function of pooled estimates may not reach the nominal level even when the number of studies in the multivariate meta‐analysis is large. In this paper, we improve the Riley method by proposing a robust variance estimator, which is asymptotically correct even when the model is misspecified (ie, when the likelihood function is incorrect). Simulation studies of a bivariate meta‐analysis, in a variety of settings, show a function of pooled estimates has improved performance when using the proposed robust variance estimator. In terms of individual pooled estimates themselves, the standard variance estimator and robust variance estimator give similar results to the original method, with appropriate coverage. The proposed robust variance estimator performs well when the number of studies is relatively large. Therefore, we recommend the use of the robust method for meta‐analyses with a relatively large number of studies (eg, m≥50). When the sample size is relatively small, we recommend the use of the robust method under the working independence assumption. We illustrate the proposed method through 2 meta‐analyses. [ABSTRACT FROM AUTHOR]

Knowledge translation is a central focus of the health research community, which includes strategies to synthesize published research to support uptake within health care practice and policy arenas. Within the literature concerning review methodologies, a new discussion has emerged concerning methods that review and synthesize published review articles. In this paper, our multidisciplinary team from family medicine, nursing, dental hygiene, kinesiology, occupational therapy, physiology, population health, clinical psychology, and library sciences contributes to this discussion by sharing our experiences in conducting 3 scoping reviews of published review studies. A brief discussion of Cochrane Collaboration overview reviews and Joanna Briggs Institute umbrella reviews foreshadows a discussion of insights from our experiences of conducting the 3 scoping reviews of published reviews. We address 6 adaptations along with our recommendations for each, which may guide other researchers with designing scoping review approaches to synthesize published reviews. The ability of researchers to publish research findings is growing, and our ability to effectively transfer findings into useful evidence for health care practice and policy is imperative to our work. [ABSTRACT FROM AUTHOR]

Multiple 'overviews of reviews' conducted on the same topic ("overlapping overviews") represent a waste of research resources and can confuse clinicians making decisions amongst competing treatments. We aimed to assess the frequency and characteristics of overlapping overviews. MEDLINE, Epistemonikos and Cochrane Database of Systematic Reviews were searched for overviews that: synthesized reviews of health interventions and conducted systematic searches. Overlap was defined as: duplication of PICO eligibility criteria, and not reported as an update nor a replication. We categorized overview topics according to 22 WHO ICD‐10 medical classifications, overviews as broad or narrow in scope, and overlap as identical, nearly identical, partial, or subsumed. Subsummation was defined as when broad overviews subsumed the populations, interventions and at least one outcome of another overview. Of 541 overviews included, 169 (31%) overlapped across similar PICO, fell within 13 WHO ICD‐10 medical classifications, and 62 topics. 148/169 (88%) overlapping overviews were broad in scope. Fifteen overviews were classified as having nearly identical overlap (9%); 123 partial overlap (73%), and 31 subsumed (18%) others. One third of overviews overlapped in content and a majority covered broad topic areas. A multiplicity of overviews on the same topic adds to the ongoing waste of research resources, time, and effort across medical disciplines. Authors of overviews can use this study and the sample of overviews to identify gaps in the evidence for future analysis, and topics that are already studied, which do not need to be duplicated. [ABSTRACT FROM AUTHOR]

Three recent papers have provided sequential methods for meta-analysis of two-treatment randomized clinical trials. This paper provides an alternate approach that has three desirable features. First, when carried out prospectively (i.e., we only have the results up to the time of our current analysis), we do not require knowledge of the information fraction (the fraction of the total information that is available at each analysis). Second, the methods work even if the expected values of the effect sizes vary from study to study. Finally, our methods have easily interpretable metrics that make sense under changing effect sizes. Although the other published methods can be adapted to be 'group sequential' (recommended), meaning that a set number and timing of looks are specified, rather than looking after every trial, ours can be used in both a continuous or group sequential manner. We provide an example on the role of probiotics in preventing necrotizing enterocolitis in preterm infants. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Visual display of reported impacts is a valuable aid to both reviewers and readers of systematic reviews. Forest plots are routinely prepared to report standardised effect sizes, but where standardised effect sizes are not available for all included studies a forest plot may misrepresent the available evidence. Tabulated data summaries to accompany the narrative synthesis can be lengthy and inaccessible. Moreover, the link between the data and the synthesis conclusions may be opaque. This paper details the preparation of visual summaries of effect direction for multiple outcomes across 29 quantitative studies of the health impacts of housing improvement. A one page summary of reported health outcomes was prepared to accompany a 10 000-word narrative synthesis. The one page summary included details of study design, internal validity, sample size, time of follow-up, as well as changes in intermediate outcomes, for example, housing condition. This approach to visually summarising complex data can aid the reviewer in cross-study analysis and improve accessibility and transparency of the narrative synthesis where standardised effect sizes are not available. Copyright © 2012 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Non-randomized studies may provide valuable evidence on the effects of interventions. They are the main source of evidence on the intended effects of some types of interventions and often provide the only evidence about the effects of interventions on long-term outcomes, rare events or adverse effects. Therefore, systematic reviews on the effects of interventions may include various types of non-randomized studies. In this second paper in a series, we address how review authors might articulate the particular non-randomized study designs they will include and how they might evaluate, in general terms, the extent to which a particular non-randomized study is at risk of important biases. We offer guidance for describing and classifying different non-randomized designs based on specific features of the studies in place of using non-informative study design labels. We also suggest criteria to consider when deciding whether to include non-randomized studies. We conclude that a taxonomy of study designs based on study design features is needed. Review authors need new tools specifically to assess the risk of bias for some non-randomized designs that involve a different inferential logic compared with parallel group trials. Copyright © 2012 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Systematic reviews of qualitative evidence have been widely used to provide information on the context and implementation of interventions, and their potential barriers and facilitators. However, such reviews face a number of methodological challenges, and there are ongoing debates as to how qualitative evidence can best be used to inform our understanding of interventions. In this paper, we use a case study of two systematic reviews of qualitative evidence on the prevention of skin cancer to explore these issues. We find that qualitative evidence not directly related to interventions is likely to be of value for such reviews, that it is often not possible to construct fully comprehensive search strategies, and that there are diminishing returns to the synthesis, in terms of added value or insight, from the inclusion of large numbers of primary studies. We conclude that there are a number of ways in which systematic reviews of qualitative evidence can be utilised in conjunction with evidence on intervention effectiveness, without compromising the rigour of the review process. In particular, the use of theory to inform frameworks for synthesis is a promising way to integrate a broader range of qualitative evidence. Copyright © 2012 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Meta-analytic structural equations modeling is increasingly used in theory testing. There has been much debate when meta-analyzed correlation matrices are used in structural equations modeling on whether to use mean observed correlations (i.e., corrected only for sampling error) or correlations corrected for study artifacts such as unreliability in measures. This paper investigates whether the fit indices are affected by the corrections and if the stability of the paths (i.e., changes in significance, magnitude, and relative strengths or rank order) is affected by the corrections. Results suggest that substantive model conclusions are generally unaffected by study artifacts and related statistical corrections as long as the variables included in the path analyses had typical levels of reliability as found in the psychological literature. More specifically, all models examined exhibited similar model fit and pathway stability. Copyright © 2011 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Effect sizes in macroeconomic are estimated by regressions on data published by statistical agencies. Funnel plots are a representation of the distribution of the resulting regression coefficients. They are normally much wider than predicted by the t-ratio of the coefficients and often asymmetric. The standard method of meta-analysts in economics assumes that the asymmetries are because of publication bias causing censoring and adjusts the average accordingly. The paper shows that some funnel asymmetries may be 'natural' so that they occur without censoring. We investigate such asymmetries by simulating funnels by pairs of data generating processes (DGPs) and estimating models (EMs), in which the EM has the problem that it disregards a property of the DGP. The problems are data dependency, structural breaks, non-normal residuals, non-linearity, and omitted variables. We show that some of these problems generate funnel asymmetries. When they do, the standard method often fails. Copyright © 2011 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

This paper presents an account of the impact that research synthesis methods, in the form of psychometric meta-analysis, has had on industrial/organizational (I/O) psychology. This paper outlines the central contributions of psychometric meta-analysis in providing a method for developing cumulative knowledge. First, this paper describes the concerns and the state of the field before the development of meta-analytic methods. Second, the paper explains how meta-analysis addressed these concerns. Third, the paper details the development of psychometric meta-analysis through VG research and describes how the use of psychometric meta-analysis spread to other topic areas in the field. Finally, the paper presents illustrative example literatures, such as training and leadership, where meta-analysis had crucial impacts. Copyright © 2011 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Understanding diagnostic test accuracy (DTA) data, especially meta-analysis of this information, can be challenging for researchers and clinicians. The use of plots of receiver-operator curve (ROC) space showing individual studies and summary estimates of diagnostic accuracy has become common but can be difficult to interpret. The assessment of heterogeneity in sensitivity and specificity across studies can be particularly difficult. In this paper, we review the key concepts of assessing DTA, starting at the level of individual studies and progressing to the setting of research synthesis. We explore the standard displays of this information and then propose and explain an alternative approach to summarizing key data. These 'cross-hairs' plots display the individual studies in ROC space with paired confidence intervals representing sensitivity and specificity, and allow for the results of meta-analysis to be overlaid on the plot. We suggest that these plots are more easily interpreted, and are a more informative graphical form than common approaches. Copyright © 2011 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

To inform health-care decision-making, treatments are often compared by synthesizing results from a number of randomized controlled trials. The meta-analysis may not only be focused on a particular pairwise comparison, but can also include multiple treatment comparisons by means of network meta-analysis. For time-to-event outcomes such as survival, pooling is typically based on the hazard ratio (HR). The proportional hazards assumption that underlies current approaches of evidence synthesis is not only often implausible, but can also have a huge impact on decisions based on differences in expected outcomes, such as cost-effectiveness analysis. The application of a constant HR implies the assumption that the treatment only has an effect on one characteristic of the survival distribution, while commonly used survival distributions, like the Weibull distribution, have both a shape and a scale parameter. Instead of using constant HRs, this paper proposes meta-analysis of treatment effects based on the shape and scale parameters of parametric survival curves. The model for meta-analysis is extended for network meta-analysis and illustrated with an example. Copyright © 2011 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

There are two popular statistical models for meta-analysis, the fixed-effect model and the random-effects model. The fact that these two models employ similar sets of formulas to compute statistics, and sometimes yield similar estimates for the various parameters, may lead people to believe that the models are interchangeable. In fact, though, the models represent fundamentally different assumptions about the data. The selection of the appropriate model is important to ensure that the various statistics are estimated correctly. Additionally, and more fundamentally, the model serves to place the analysis in context. It provides a framework for the goals of the analysis as well as for the interpretation of the statistics. In this paper we explain the key assumptions of each model, and then outline the differences between the models. We conclude with a discussion of factors to consider when choosing between the two models. Copyright © 2010 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

The presence of outliers and influential cases may affect the validity and robustness of the conclusions from a meta-analysis. While researchers generally agree that it is necessary to examine outlier and influential case diagnostics when conducting a meta-analysis, limited studies have addressed how to obtain such diagnostic measures in the context of a meta-analysis. The present paper extends standard diagnostic procedures developed for linear regression analyses to the meta-analytic fixed- and random/mixed-effects models. Three examples are used to illustrate the usefulness of these procedures in various research settings. Issues related to these diagnostic procedures in meta-analysis are also discussed. Copyright © 2010 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Synthesizers of evidence are increasingly likely to encounter studies published in predatory journals during the evidence synthesis process. The evidence synthesis discipline is uniquely positioned to encounter novel concerns associated with predatory journals. The objective of this research was to explore the attitudes, opinions, and experiences of experts in the synthesis of evidence regarding predatory journals. Employing a descriptive survey-based cross-sectional study design, these experts were asked a series of questions regarding predatory journals to explore these attitudes, opinions, and experiences. Two hundred and sixty four evidence synthesis experts responded to this survey. Most respondents agreed with the definition of a predatory journal (86%), however several (19%) responded that this definition was difficult to apply practically. Many respondents believed that studies published in predatory journals are still eligible for inclusion into an evidence synthesis project. However, this was only after the study had been determined to be 'high-quality' (39%) or if the results were validated (13%). While many respondents could identify common characteristics of these journals, there was still hesitancy regarding the appropriate methods to follow when considering including these studies into an evidence synthesis project., (© 2022 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.)

Citation tracking (CT) collects references with citation relationships to pertinent references that are already known. This scoping review maps the benefit of and the tools and terminology used for CT in health-related systematic literature searching. We included methodological studies on evidence retrieval by CT in health-related literature searching without restrictions on study design, language, or publication date. We searched MEDLINE/Ovid, Web of Science Core Collection, CINAHL/EBSCOhost, LLISFT/EBSCOhost, LISTA/EBSCOhost, conducted web searching via Google Scholar, backward/forward CT of included studies and pertinent reviews, and contacting of experts. Two reviewers independently assessed eligibility. Data extraction and analysis were performed by one reviewer and checked by another. We screened 11,861 references and included 47 studies published between 1985 and 2021. Most studies (96%) assessed the benefit of CT either as supplementary or primary/stand-alone search method. Added value of CT for evidence retrieval was found by 96% of them. Science Citation Index and Social Sciences Citation Index were the most common citation indexes used. Application of multiple citation indexes in parallel, co-citing or co-cited references, CT iterations, or software tools was rare. CT terminology was heterogeneous and frequently ambiguous. The use of CT showed an added value in most of the identified studies; however, the benefit of CT in health-related systematic literature searching likely depends on multiple factors that could not be assessed with certainty. Application, terminology, and reporting are heterogeneous. Based on our results, we plan a Delphi study to develop recommendations for the use and reporting of CT., (© 2023 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.)

Authors and indexers are increasingly including terms for adverse drug effects in the titles, abstracts, or indexing of records in MEDLINE and Embase. However, it is not clear if this is the same for studies with nondrug adverse effects data. We therefore assessed the feasibility of using adverse effects terms when searching MEDLINE or Embase to retrieve papers of nondrug adverse effects. A collection of papers that reported data on nondrug adverse effects was sought from included studies of systematic reviews. Each included study was analysed to ascertain whether the corresponding record in MEDLINE and Embase included adverse effects terms in the title, abstract, or indexing. From 9129 records screened from DARE, 30 reviews evaluating nondrug adverse effects met our inclusion criteria. From these, 635 unique papers were included in our analysis. Sensitive searches for adverse effects required generic and specific named adverse effects terms using the title, abstract, and indexing. Records relating to surgical interventions were more likely to contain adverse effects terms than records relating to nonsurgical interventions. Using any adverse effects terms in the title, abstract or indexing in MEDLINE and Embase would have identified an average of 94% of papers on surgical adverse effect interventions per systematic review and 72% of papers on nonsurgical adverse effects per systematic review. Hence, while a generic nondrug adverse effect search filter may not yet be feasible, a filter for the adverse effects of surgical interventions may be within reach. [ABSTRACT FROM AUTHOR]

Background Many reviews aim to compare numerous treatments and report results stratified by subgroups (eg, by disease severity). In such cases, a network meta-analysis model including treatment by covariate interactions can estimate the relative effects of all treatment pairings for each subgroup of patients. Two key assumptions underlie such models: consistency of treatment effects and consistency of the regression coefficients for the interactions. Consistency may differ depending on the covariate value at which consistency is assessed. For valid inference, we need to be confident of consistency for the relevant range of covariate values. In this paper, we demonstrate how to assess consistency of treatment effects from direct and indirect evidence at various covariate values. Methods Consistency is assessed using visual inspection, inconsistency estimates, and probabilities. The method is applied to an individual patient dataset comparing artemisinin combination therapies for treating uncomplicated malaria in children using the covariate age. Results The magnitude of the inconsistency appears to be decreasing with increasing age for each comparison. For one comparison, direct and indirect evidence differ for age 1 ( P = .05), and this brings results for age 1 for all comparisons into question. Conclusion When fitting models including interactions, the consistency of direct and indirect evidence must be assessed across the range of covariates included in the trials. Clinical inferences are only valid for covariate values for which results are consistent. [ABSTRACT FROM AUTHOR]

Clinical trials have different designs: In late stage drug development, the parallel trial design is the most frequent one; however, the crossover design is not rare; different techniques are used to analyse their results. Although both designs measure the same treatment effect, combining parallel and crossover trials in a meta-analysis is not straightforward. We present here a meta-analysis method based on generalised estimating equation (GEE) regression to combine aggregated results of crossover and parallel trials using a marginal estimation approach. This method is based on the fixed effects meta-analytic model; it allows combining average outcomes belonging to the exponential distributions obtained from trials of different designs and in particular from crossover trials with more than 2 periods and 2 treatments. By extending the methods proposed so far to combine the 2 trial designs, the GEE regression allows for adjusting for bias, such as a carry-over effect typical in crossover trials. In this paper, the GEE meta-analysis method is compared to the classical weighted average method with examples of published and simulated meta-analyses. Although the GEE can account for crossover specificities, it is limited by the availability of detailed trial information often encountered with reports of these trials. [ABSTRACT FROM AUTHOR]

Network meta-analysis (NMA) combines direct and indirect evidence comparing more than 2 treatments. Inconsistency arises when these 2 information sources differ. Previous work focuses on inconsistency detection, but little has been done on how to proceed after identifying inconsistency. The key issue is whether inconsistency changes an NMA's substantive conclusions. In this paper, we examine such discrepancies from a diagnostic point of view. Our methods seek to detect influential and outlying observations in NMA at a trial-by-arm level. These observations may have a large effect on the parameter estimates in NMA, or they may deviate markedly from other observations. We develop formal diagnostics for a Bayesian hierarchical model to check the effect of deleting any observation. Diagnostics are specified for generalized linear hierarchical NMA models and investigated for both published and simulated datasets. Results from our example dataset using either contrast- or arm-based models and from the simulated datasets indicate that the sources of inconsistency in NMA tend not to be influential, though results from the example dataset suggest that they are likely to be outliers. This mimics a familiar result from linear model theory, in which outliers with low leverage are not influential. Future extensions include incorporating baseline covariates and individual-level patient data. [ABSTRACT FROM AUTHOR]

When conducting research synthesis, the collection of studies that will be combined often do not measure the same set of variables, which creates missing data. When the studies to combine are longitudinal, missing data can occur on the observation-level (time-varying) or the subject-level (non-time-varying). Traditionally, the focus of missing data methods for longitudinal data has been on missing observation-level variables. In this paper, we focus on missing subject-level variables and compare two multiple imputation approaches: a joint modeling approach and a sequential conditional modeling approach. We find the joint modeling approach to be preferable to the sequential conditional approach, except when the covariance structure of the repeated outcome for each individual has homogenous variance and exchangeable correlation. Specifically, the regression coefficient estimates from an analysis incorporating imputed values based on the sequential conditional method are attenuated and less efficient than those from the joint method. Remarkably, the estimates from the sequential conditional method are often less efficient than a complete case analysis, which, in the context of research synthesis, implies that we lose efficiency by combining studies. Copyright © 2015 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Temporal changes in magnitude of effect sizes reported in many areas of research are a threat to the credibility of the results and conclusions of meta-analysis. Numerous sequential methods for meta-analysis have been proposed to detect changes and monitor trends in effect sizes so that meta-analysis can be updated when necessary and interpreted based on the time it was conducted. The difficulties of sequential meta-analysis under the random-effects model are caused by dependencies in increments introduced by the estimation of the heterogeneity parameter τ2. In this paper, we propose the use of a retrospective cumulative sum (CUSUM)-type test with bootstrap critical values. This method allows retrospective analysis of the past trajectory of cumulative effects in random-effects meta-analysis and its visualization on a chart similar to CUSUM chart. Simulation results show that the new method demonstrates good control of Type I error regardless of the number or size of the studies and the amount of heterogeneity. Application of the new method is illustrated on two examples of medical meta-analyses. © 2016 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd. [ABSTRACT FROM AUTHOR]

I reflect on the evolution of meta-analysis in clinical research over 35 years since I first encountered the key ideas in the late 1970s. Copyright © 2015 Douglas G. Altman. [ABSTRACT FROM AUTHOR]

Background: The CLUSTER model of searching was proposed as a systematic method of searching for studies for reviews of complex interventions. Aim: The method has not been evaluated before. This methodological review identified and evaluated a sample of evidence syntheses that have used CLUSTER. Methods: A forward citation search on the seed CLUSTER publication was conducted on Web of Science Core Collection using six journal citation indexes and Google Scholar in December 2020. Reviews which used the CLUSTER method were eligible for inclusion. A narrative synthesis was used to describe the types of evidence syntheses that used CLUSTER searching, the extent to which the CLUSTER approach has been operationalised within evidence syntheses and whether the value, benefits and limitations of CLUSTER were assessed by the reviewers. Findings: A total of 16 reviews were identified and eligible for synthesis. Six different review types that used CLUSTER were identified with realist reviews being the most prominent. The evaluation of complex interventions was the most common review topic area. The use of CLUSTER varied among reviews with the retrieval of sibling studies being the most common reason. 'Citations' and 'Lead authors' were the most followed elements of CLUSTER. Conclusions: Evidence suggests that CLUSTER has been adopted for use in reviews of complex interventions. Its usage varied among the included reviews. It is imperative that future reviewers diligently report the elements and steps of CLUSTER that were utilised in order to provide a reproducible and transparent search strategy that can be reported with similar transparency to bibliographic database searches. [ABSTRACT FROM AUTHOR]

Background: Enhanced uptake of systematic reviews that use qualitative comparative analyses (QCA) requires knowing how end-users interpret such findings. The study purpose was to identify effective approaches to communicating results from a QCA within a systematic review., Methods: Sequential exploratory mixed methods design; thematic analysis of interviews with 11 end-users followed by a randomized experiment with 254 participants that provided QCA results for a hypothetical review presented through three formats (text, table, and figure). A survey administered after the experiment assessed subjective and objective comprehension of QCA results., Results: Interview themes included use of jargon; appropriate use of appendices, tables, figures; and integration of QCA results within the systematic review. In the experiment, we observed a significant difference (p = 0.035) in subjective comprehension across the three presentation formats. Participants randomized to the figure and text formats scored higher compared to the table. No significant differences were observed for objective comprehension overall (p = 0.11). However, for parameter interpretation (a unique component of QCA results), scores among participants that received the figure format were significantly higher than scores for participants who received the text (p = 0.001) or table (p = 0.004). No significant differences (p = 0.09) were observed in objective comprehension for configuration interpretation., Conclusions: End-users of systematic reviews saw value in the use of QCA, but unfamiliar methods and terminology were barriers to full understanding of the findings. When presenting results, a figure format appears to be superior to text or table formats based on measures of subjective comprehension and some measures of objective comprehension., (© 2022 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.)

Descriptive analyses of socially important or theoretically interesting phenomena and trends are a vital component of research in the behavioral, social, economic, and health sciences. Such analyses yield reliable results when using representative individual participant data (IPD) from studies with complex survey designs, including educational large-scale assessments (ELSAs) or social, health, and economic survey and panel studies. The meta-analytic integration of these results offers unique and novel research opportunities to provide strong empirical evidence of the consistency and generalizability of important phenomena and trends. Using ELSAs as an example, this tutorial offers methodological guidance on how to use the two-stage approach to IPD meta-analysis to account for the statistical challenges of complex survey designs (e.g., sampling weights, clustered and missing IPD), first, to conduct descriptive analyses (Stage 1), and second, to integrate results with three-level meta-analytic and meta-regression models to take into account dependencies among effect sizes (Stage 2). The two-stage approach is illustrated with IPD on reading achievement from the Programme for International Student Assessment (PISA). We demonstrate how to analyze and integrate standardized mean differences (e.g., gender differences), correlations (e.g., with students' socioeconomic status [SES]), and interactions between individual characteristics at the participant level (e.g., the interaction between gender and SES) across several PISA cycles. All the datafiles and R scripts we used are available online. Because complex social, health, or economic survey and panel studies share many methodological features with ELSAs, the guidance offered in this tutorial is also helpful for synthesizing research evidence from these studies., (© 2022 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.)

Meta-analysis is a popular and flexible analysis that can be fit in many modeling frameworks. Two methods of fitting meta-analyses that are growing in popularity are structural equation modeling (SEM) and multilevel modeling (MLM). By using SEM or MLM to fit a meta-analysis researchers have access to powerful techniques associated with SEM and MLM. This paper details how to use one such technique, multiple group analysis, to test categorical moderators in meta-analysis. In a multiple group meta-analysis a model is fit to each level of the moderator simultaneously. By constraining parameters across groups any model parameter can be tested for equality. Using multiple groups to test for moderators is especially relevant in random-effects meta-analysis where both the mean and the between studies variance of the effect size may be compared across groups. A simulation study and the analysis of a real data set are used to illustrate multiple group modeling with both SEM and MLM. Issues related to multiple group meta-analysis and future directions for research are discussed. [ABSTRACT FROM AUTHOR]

This paper investigates how inconsistency (as measured by the I2 statistic) among studies in a meta-analysis may differ, according to the type of outcome data and effect measure. We used hierarchical models to analyse data from 3873 binary, 5132 continuous and 880 mixed outcome meta-analyses within the Cochrane Database of Systematic Reviews. Predictive distributions for inconsistency expected in future meta-analyses were obtained, which can inform priors for between-study variance. Inconsistency estimates were highest on average for binary outcome meta-analyses of risk differences and continuous outcome meta-analyses. For a planned binary outcome meta-analysis in a general research setting, the predictive distribution for inconsistency among log odds ratios had median 22% and 95% CI: 12% to 39%. For a continuous outcome meta-analysis, the predictive distribution for inconsistency among standardized mean differences had median 40% and 95% CI: 15% to 73%. Levels of inconsistency were similar for binary data measured by log odds ratios and log relative risks. Fitted distributions for inconsistency expected in continuous outcome meta-analyses using mean differences were almost identical to those using standardized mean differences. The empirical evidence on inconsistency gives guidance on which outcome measures are most likely to be consistent in particular circumstances and facilitates Bayesian meta-analysis with an informative prior for heterogeneity. [ABSTRACT FROM AUTHOR]

Structural equation models play an important role in the social sciences. Consequently, there is an increasing use of meta-analytic methods to combine evidence from studies that estimate the parameters of structural equation models. Two approaches are used to combine evidence from structural equation models: A direct approach that combines structural coefficients and an indirect approach that first combines correlation matrices and estimates structural coefficients from the combined correlation matrix. When there is no heterogeneity across studies, direct estimates of structural coefficients from several studies is an appealing approach. Heterogeneity of correlation matrices across studies presents both practical and conceptual problems. An alternative approach to heterogeneity is suggested as an example of how to better handle heterogeneity in this context. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Meta-analytic structural equation modeling (MASEM) combines the techniques of meta-analysis and structural equation modeling for the purpose of synthesizing correlation or covariance matrices and fitting structural equation models on the pooled correlation or covariance matrix. Both fixed-effects and random-effects models can be defined in MASEM. Random-effects models are well known in conventional meta-analysis but are less studied in MASEM. The primary objective of this paper was to address issues related to random-effects models in MASEM. Specifically, we compared two different random-effects models in MASEM-correlation-based MASEM and parameter-based MASEM-and explored their strengths and limitations. Two examples were used to illustrate the similarities and differences between these models. We offered some practical guidelines for choosing between these two models. Future directions for research on random-effects models in MASEM were also discussed. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Meta-analytic structural equation modeling (MASEM) involves fitting models to a common population correlation matrix that is estimated on the basis of correlation coefficients that are reported by a number of independent studies. MASEM typically consist of two stages. The method that has been found to perform best in terms of statistical properties is the two-stage structural equation modeling, in which maximum likelihood analysis is used to estimate the common correlation matrix in the first stage, and weighted least squares analysis is used to fit structural equation models to the common correlation matrix in the second stage. In the present paper, we propose an alternative method, ML MASEM, that uses ML estimation throughout. In a simulation study, we use both methods and compare chi-square distributions, bias in parameter estimates, false positive rates, and true positive rates. Both methods appear to yield unbiased parameter estimates and false and true positive rates that are close to the expected values. ML MASEM parameter estimates are found to be significantly less bias than two-stage structural equation modeling estimates, but the differences are very small. The choice between the two methods may therefore be based on other fundamental or practical arguments. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Bayesian statistical approaches to mixed treatment comparisons (MTCs) are becoming more popular because of their flexibility and interpretability. Many randomized clinical trials report multiple outcomes with possible inherent correlations. Moreover, MTC data are typically sparse (although richer than standard meta-analysis, comparing only two treatments), and researchers often choose study arms based upon which treatments emerge as superior in previous trials. In this paper, we summarize existing hierarchical Bayesian methods for MTCs with a single outcome and introduce novel Bayesian approaches for multiple outcomes simultaneously, rather than in separate MTC analyses. We do this by incorporating partially observed data and its correlation structure between outcomes through contrast-based and arm-based parameterizations that consider any unobserved treatment arms as missing data to be imputed. We also extend the model to apply to all types of generalized linear model outcomes, such as count or continuous responses. We offer a simulation study under various missingness mechanisms (e.g., missing completely at random, missing at random, and missing not at random) providing evidence that our models outperform existing models in terms of bias, mean squared error, and coverage probability then illustrate our methods with a real MTC dataset. We close with a discussion of our results, several contentious issues in MTC analysis, and a few avenues for future methodological development. Copyright © 2015 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Network meta-analysis enables the simultaneous synthesis of a network of clinical trials comparing any number of treatments. Potential inconsistencies between estimates of relative treatment effects are an important concern, and several methods to detect inconsistency have been proposed. This paper is concerned with the node-splitting approach, which is particularly attractive because of its straightforward interpretation, contrasting estimates from both direct and indirect evidence. However, node-splitting analyses are labour-intensive because each comparison of interest requires a separate model. It would be advantageous if node-splitting models could be estimated automatically for all comparisons of interest. We present an unambiguous decision rule to choose which comparisons to split, and prove that it selects only comparisons in potentially inconsistent loops in the network, and that all potentially inconsistent loops in the network are investigated. Moreover, the decision rule circumvents problems with the parameterisation of multi-arm trials, ensuring that model generation is trivial in all cases. Thus, our methods eliminate most of the manual work involved in using the node-splitting approach, enabling the analyst to focus on interpreting the results. © 2015 The Authors Research Synthesis Methods Published by John Wiley & Sons Ltd. [ABSTRACT FROM AUTHOR]

Clinical trials registers form an important part of the search for studies in systematic reviews of intervention effectiveness but the search interfaces and functionality of registers can be challenging to search systematically and resource intensive to search well. We report a technical review of the search interfaces of three leading trials register resources: ClinicalTrials.gov, the EU Clinical Trials Register and the WHO International Clinical Trials Registers Platform. The technical review used a validated checklist to identify areas where the search interfaces of these trials register resources performed well, where performance was adequate, where performance was poor, and to identify differences between search interfaces. The review found low overall scores for each of the interfaces (ClinicalTrials.gov 55/165, the EU Clinical Trials Register 25/165, the WHO International Clinical Trials Registers Platform 32/165). This finding suggests a need for joined‐up dialogue between the producers of the registers and researchers who search them via these interfaces. We also set out a series of four proposed changes which might improve the search interfaces. Trials registers are an invaluable resource in systematic reviews of intervention effectiveness. With the continued growth in systematic reviews, and initiatives such as 'AllTrials', there is an anticipated need for these resources. We conclude that small changes to the search interfaces, and improved dialogue with providers, might improve the future search functionality of these valuable resources. [ABSTRACT FROM AUTHOR]

The short time frame associated with rapid reviews can be challenging for researchers conducting qualitative evidence synthesis. In these circumstances a Best‐Fit Framework Synthesis, drawing upon existing theory and/or research, may be conducted to rapidly make sense of qualitative evidence. This article discusses a "Rapid Best‐Fit" approach to conducting Framework Synthesis within an 6‐week rapid systematic review of qualitative evidence. In the absence of a suitable theoretical model to inform the synthesis, we drew upon our research objectives to structure a framework, and to identify the studies which contained the most relevant data. Themes from these studies were used to revise the initial framework before inductive thematic synthesis finalized theme content. This "Rapid Best‐Fit" approach yielded results aligned with the needs of the commissioners of the review and is a useful addition to qualitative synthesis methods, allowing for the incorporation of diverse views and experiences into fast‐paced decision making scenarios. [ABSTRACT FROM AUTHOR]