Search

Interrupted time series (ITS) studies are frequently used to examine the impact of population-level interventions or exposures. Systematic reviews with meta-analyses including ITS designs may inform public health and policy decision-making. Re-analysis of ITS may be required for inclusion in meta-analysis. While publications of ITS rarely provide raw data for re-analysis, graphs are often included, from which time series data can be digitally extracted. However, the accuracy of effect estimates calculated from data digitally extracted from ITS graphs is currently unknown. Forty-three ITS with available datasets and time series graphs were included. Time series data from each graph was extracted by four researchers using digital data extraction software. Data extraction errors were analysed. Segmented linear regression models were fitted to the extracted and provided datasets, from which estimates of immediate level and slope change (and associated statistics) were calculated and compared across the datasets. Although there were some data extraction errors of time points, primarily due to complications in the original graphs, they did not translate into important differences in estimates of interruption effects (and associated statistics). Using digital data extraction to obtain data from ITS graphs should be considered in reviews including ITS. Including these studies in meta-analyses, even with slight inaccuracy, is likely to outweigh the loss of information from non-inclusion.

Interrupted time series (ITS) are often meta-analysed to inform public health and policy decisions but examination of the statistical methods for ITS analysis and meta-analysis in this context is limited. We simulated meta-analyses of ITS studies with continuous outcome data, analysed the studies using segmented linear regression with two estimation methods [ordinary least squares (OLS) and restricted maximum likelihood (REML)], and meta-analysed the immediate level- and slope-change effect estimates using fixed-effect and (multiple) random-effects meta-analysis methods. Simulation design parameters included varying series length; magnitude of lag-1 autocorrelation; magnitude of level- and slope-changes; number of included studies; and, effect size heterogeneity. All meta-analysis methods yielded unbiased estimates of the interruption effects. All random effects meta-analysis methods yielded coverage close to the nominal level, irrespective of the ITS analysis method used and other design parameters. However, heterogeneity was frequently overestimated in scenarios where the ITS study standard errors were underestimated, which occurred for short series or when the ITS analysis method did not appropriately account for autocorrelation. The performance of meta-analysis methods depends on the design and analysis of the included ITS studies. Although all random effects methods performed well in terms of coverage, irrespective of the ITS analysis method, we recommend the use of effect estimates calculated from ITS methods that adjust for autocorrelation when possible. Doing so will likely to lead to more accurate estimates of the heterogeneity variance.

Systematic reviews should be reported completely and accurately so that users can interpret and appraise the review findings appropriately. In this commentary, we describe key features of a major update of the Preferred Reporting Items for Systematic reviews and Meta-Analyses statement--PRISMA 2020--and discuss its implications for research synthesis methodologists.

Introduction: Interrupted Time Series (ITS) studies may be used to assess the impact of an interruption, such as an intervention or exposure. The data from such studies are particularly amenable to visual display and, when clearly depicted, can readily show the short- and long-term impact of an interruption. Further, well-constructed graphs allow data to be extracted using digitizing software, which can facilitate their inclusion in systematic reviews and meta-analyses. Aim: We provide recommendations for graphing ITS data, examine the properties of plots presented in ITS studies, and provide examples employing our recommendations. Methods and results: Graphing recommendations from seminal data visualization resources were adapted for use with ITS studies. The adapted recommendations cover plotting of data points, trend lines, interruptions, additional lines and general graph components. We assessed whether 217 graphs from recently published (2013-2017) ITS studies met our recommendations and found that 130 graphs (60%) had clearly distinct data points, 100 (46%) had trend lines, and 161 (74%) had a clearly defined interruption. Accurate data extraction (requiring distinct points that align with axis tick marks and labels that allow the points to be interpreted) was possible in only 72 (33%) graphs. Conclusion: We found that many ITS graphs did not meet our recommendations and could be improved with simple changes. Our proposed recommendations aim to achieve greater standardization and improvement in the display of ITS data, and facilitate re-use of the data in systematic reviews and meta-analyses.

Background: Interrupted time series (ITS) studies contribute importantly to systematic reviews of population‐level interventions. We aimed to develop and validate search filters to retrieve ITS studies in MEDLINE and PubMed. Methods: A total of 1017 known ITS studies (published 2013–2017) were analysed using text mining to generate candidate terms. A control set of 1398 time‐series studies were used to select differentiating terms. Various combinations of candidate terms were iteratively tested to generate three search filters. An independent set of 700 ITS studies was used to validate the filters' sensitivities. The filters were test‐run in Ovid MEDLINE and the records randomly screened for ITS studies to determine their precision. Finally, all MEDLINE filters were translated to PubMed format and their sensitivities in PubMed were estimated. Results: Three search filters were created in MEDLINE: a precision‐maximising filter with high precision (78%; 95% CI 74%–82%) but moderate sensitivity (63%; 59%–66%), most appropriate when there are limited resources to screen studies; a sensitivity‐and‐precision‐maximising filter with higher sensitivity (81%; 77%–83%) but lower precision (32%; 28%–36%), providing a balance between expediency and comprehensiveness; and a sensitivity‐maximising filter with high sensitivity (88%; 85%–90%) but likely very low precision, useful when combined with specific content terms. Similar sensitivity estimates were found for PubMed versions. Conclusion: Our filters strike different balances between comprehensiveness and screening workload and suit different research needs. Retrieval of ITS studies would be improved if authors identified the ITS design in the titles. [ABSTRACT FROM AUTHOR]

We aimed to explore, in a sample of systematic reviews (SRs) with meta‐analyses of the association between food/diet and health‐related outcomes, whether systematic reviewers selectively included study effect estimates in meta‐analyses when multiple effect estimates were available. We randomly selected SRs of food/diet and health‐related outcomes published between January 2018 and June 2019. We selected the first presented meta‐analysis in each review (index meta‐analysis), and extracted from study reports all study effect estimates that were eligible for inclusion in the meta‐analysis. We calculated the Potential Bias Index (PBI) to quantify and test for evidence of selective inclusion. The PBI ranges from 0 to 1; values above or below 0.5 suggest selective inclusion of effect estimates more or less favourable to the intervention, respectively. We also compared the index meta‐analytic estimate to the median of a randomly constructed distribution of meta‐analytic estimates (i.e., the estimate expected when there is no selective inclusion). Thirty‐nine SRs with 312 studies were included. The estimated PBI was 0.49 (95% CI 0.42–0.55), suggesting that the selection of study effect estimates from those reported was consistent with a process of random selection. In addition, the index meta‐analytic effect estimates were similar, on average, to what we would expect to see in meta‐analyses generated when there was no selective inclusion. Despite this, we recommend that systematic reviewers report the methods used to select effect estimates to include in meta‐analyses, which can help readers understand the risk of selective inclusion bias in the SRs. [ABSTRACT FROM AUTHOR]