Search

When performing a systematic review, researchers screen the articles retrieved after a broad search strategy one by one, which is time-consuming. Computerised support of this screening process has been applied with varying success. This is partly due to the dependency on large amounts of data to develop models that predict inclusion. In this paper, we present an approach to choose which data to use in model training and compare it with established approaches. We used a dataset of 50 Cochrane diagnostic test accuracy reviews, and each was used as a target review. From the remaining 49 reviews, we selected those that most closely resembled the target review's clinical topic using the cosine similarity metric. Included and excluded studies from these selected reviews were then used to develop our prediction models. The performance of models trained on the selected reviews was compared against models trained on studies from all available reviews. The prediction models performed best with a larger number of reviews in the training set and on target reviews that had a research subject similar to other reviews in the dataset. Our approach using cosine similarity may reduce computational costs for model training and the duration of the screening process.

Comparative accuracy studies evaluate the relative performance of two or more diagnostic tests. As any other form of research, such studies should be reported in an informative manner, to allow replication and to be useful for decision-making. In this study we aimed to assess whether and how components of test comparisons were reported in comparative accuracy studies. We evaluated 100 comparative accuracy studies, published in 2015, 2016 or 2017, randomly sampled from 238 comparative accuracy systematic reviews. We extracted information on 20 reporting items, pertaining to the identification of the test comparison, its validity, and the actual results of the comparison. About a third of the studies (n = 36) did not report the comparison as a study objective or hypothesis. Although most studies (n = 86) reported how participants had been allocated to index tests, we could often not evaluate whether test interpreters had been blinded to the results of other index tests (n = 40; among 59 applicable studies), nor could we identify the sequence of index tests (n = 52; among 90 applicable studies) or the methods for comparing measures of accuracy (n = 59). Two-by-four table data (revealing the agreement between index tests) were only reported by 9 of 90 paired comparative studies. More than half of the studies (n = 64) did not provide measures of statistical imprecision for comparative accuracy. Our findings suggest that components of test comparisons are frequently missing or incompletely described in comparative accuracy studies included in systematic reviews. Explicit guidance for reporting comparative accuracy studies may facilitate the production of full and informative study reports.

Background: Individual patient data meta-analyses (IPD-MA) are regarded as the gold standard for systematic reviews, which also applies to systematic reviews of diagnostic test accuracy (DTA) studies. An increasing number of DTA systematic reviews with IPD-MA have been published in recent years, but there is much variation in how these IPD-MA were performed. A number of existing methods were found, but there is no consensus as to which methods are preferred as the standard methods for statistical analysis in DTA IPD-MA. Objectives: To create a web-based tool which integrates recommended statistical analyses for DTA IPD-MA, and allows researchers to analyse the data and visualize the results with interactive plots. Methods: A systematic methodological review was performed to identify statistical analyses and data visualization methods used in DTA IPD-MA. Methods were evaluated by the authors and recommended analyses were integrated into the IPDmada tool which is freely available online with the user interface developed with R Shiny package. Results: IPDmada allows users to upload their own data, perform the meta-analysis with both continuous and dichotomized tests, and incorporate individual level covariate-adjusted analysis. All tables and figures can be exported as .csv or .pdf files. A hypothetical dataset was used to illustrate the application of IPDmada. Conclusions: IPDmada will be very helpful to researchers doing DTA IPD-MA, since it not only facilitates the statistical analysis but also provides a standard framework. The introduction of IPDmada will harmonize the methods used in DTA IPD-MA and ensure the quality of such analyses.

Background: Preferential publication of studies with positive findings can lead to overestimation of diagnostic test accuracy (i.e. publication bias). Understanding the contribution of the editorial process to publication bias could inform interventions to optimize the evidence guiding clinical decisions. Purpose/Hypothesis: To evaluate whether accuracy estimates, abstract conclusion positivity, and completeness of abstract reporting are associated with acceptance to radiology conferences and journals. Study Type: Meta‐research. Population: Abstracts submitted to radiology conferences (European Society of Gastrointestinal and Abdominal Radiology (ESGAR) and International Society for Magnetic Resonance in Medicine (ISMRM)) from 2008 to 2018 and manuscripts submitted to radiology journals (Radiology, Journal of Magnetic Resonance Imaging [JMRI]) from 2017 to 2018. Primary clinical studies evaluating sensitivity and specificity of a diagnostic imaging test in humans with available editorial decisions were included. Assessment: Primary variables (Youden's index [YI > 0.8 vs. <0.8], abstract conclusion positivity [positive vs. neutral/negative], number of reported items on the Standards for Reporting of Diagnostic Accuracy Studies [STARD] for Abstract guideline) and confounding variables (prospective vs. retrospective/unreported, sample size, study duration, interobserver agreement assessment, subspecialty, modality) were extracted. Statistical Tests: Multivariable logistic regression to obtain adjusted odds ratio (OR) as a measure of the association between the primary variables and acceptance by radiology conferences and journals; 95% confidence intervals (CIs) and P‐values were obtained; the threshold for statistical significance was P < 0.05. Results: A total of 1000 conference abstracts (500 ESGAR and 500 ISMRM) and 1000 journal manuscripts (505 Radiology and 495 JMRI) were included. Conference abstract acceptance was not significantly associated with YI (adjusted OR = 0.97 for YI > 0.8; CI = 0.70–1.35), conclusion positivity (OR = 1.21 for positive conclusions; CI = 0.75–1.90) or STARD for Abstracts adherence (OR = 0.96 per unit increase in reported items; CI = 0.82–1.18). Manuscripts with positive abstract conclusions were less likely to be accepted by radiology journals (OR = 0.45; CI = 0.24–0.86), while YI (OR = 0.85; CI = 0.56–1.29) and STARD for Abstracts adherence (OR = 1.06; CI = 0.87–1.30) showed no significant association. Positive conclusions were present in 86.7% of submitted conference abstracts and 90.2% of journal manuscripts. Data Conclusion: Diagnostic test accuracy studies with positive findings were not preferentially accepted by the evaluated radiology conferences or journals. Evidence Level: 3 Technical Efficacy: Stage 2. [ABSTRACT FROM AUTHOR]

When performing a systematic review, researchers screen the articles retrieved after a broad search strategy one by one, which is time‐consuming. Computerised support of this screening process has been applied with varying success. This is partly due to the dependency on large amounts of data to develop models that predict inclusion. In this paper, we present an approach to choose which data to use in model training and compare it with established approaches. We used a dataset of 50 Cochrane diagnostic test accuracy reviews, and each was used as a target review. From the remaining 49 reviews, we selected those that most closely resembled the target review's clinical topic using the cosine similarity metric. Included and excluded studies from these selected reviews were then used to develop our prediction models. The performance of models trained on the selected reviews was compared against models trained on studies from all available reviews. The prediction models performed best with a larger number of reviews in the training set and on target reviews that had a research subject similar to other reviews in the dataset. Our approach using cosine similarity may reduce computational costs for model training and the duration of the screening process. [ABSTRACT FROM AUTHOR]

Comparative accuracy studies evaluate the relative performance of two or more diagnostic tests. As any other form of research, such studies should be reported in an informative manner, to allow replication and to be useful for decision‐making. In this study we aimed to assess whether and how components of test comparisons were reported in comparative accuracy studies. We evaluated 100 comparative accuracy studies, published in 2015, 2016 or 2017, randomly sampled from 238 comparative accuracy systematic reviews. We extracted information on 20 reporting items, pertaining to the identification of the test comparison, its validity, and the actual results of the comparison. About a third of the studies (n = 36) did not report the comparison as a study objective or hypothesis. Although most studies (n = 86) reported how participants had been allocated to index tests, we could often not evaluate whether test interpreters had been blinded to the results of other index tests (n = 40; among 59 applicable studies), nor could we identify the sequence of index tests (n = 52; among 90 applicable studies) or the methods for comparing measures of accuracy (n = 59). Two‐by‐four table data (revealing the agreement between index tests) were only reported by 9 of 90 paired comparative studies. More than half of the studies (n = 64) did not provide measures of statistical imprecision for comparative accuracy. Our findings suggest that components of test comparisons are frequently missing or incompletely described in comparative accuracy studies included in systematic reviews. Explicit guidance for reporting comparative accuracy studies may facilitate the production of full and informative study reports. [ABSTRACT FROM AUTHOR]

Background: Individual patient data meta‐analyses (IPD‐MA) are regarded as the gold standard for systematic reviews, which also applies to systematic reviews of diagnostic test accuracy (DTA) studies. An increasing number of DTA systematic reviews with IPD‐MA have been published in recent years, but there is much variation in how these IPD‐MA were performed. A number of existing methods were found, but there is no consensus as to which methods are preferred as the standard methods for statistical analysis in DTA IPD‐MA. Objectives: To create a web‐based tool which integrates recommended statistical analyses for DTA IPD‐MA, and allows researchers to analyse the data and visualize the results with interactive plots. Methods: A systematic methodological review was performed to identify statistical analyses and data visualization methods used in DTA IPD‐MA. Methods were evaluated by the authors and recommended analyses were integrated into the IPDmada tool which is freely available online with the user interface developed with R Shiny package. Results: IPDmada allows users to upload their own data, perform the meta‐analysis with both continuous and dichotomized tests, and incorporate individual level covariate‐adjusted analysis. All tables and figures can be exported as.csv or.pdf files. A hypothetical dataset was used to illustrate the application of IPDmada. Conclusions: IPDmada will be very helpful to researchers doing DTA IPD‐MA, since it not only facilitates the statistical analysis but also provides a standard framework. The introduction of IPDmada will harmonize the methods used in DTA IPD‐MA and ensure the quality of such analyses. Highlights: IPDmada is a newly developed web‐based tool for performing statistical analysis of individual patient data meta‐analysis of diagnostic accuracy and visualizing the results.The tool is freely available to all the researchers, and requiring no installation of statistical software/packages.The tool has an user‐friendly interface, and allows meta‐analysis on both dichotomized and continuous test results. Researchers can easily use this tool to investigate the threshold effect and covariate effect on the summary accuracy.The introduction and implementation of IPDmada will serve as a useful tool for DTA IPD‐MA and increase the quality of such studies. [ABSTRACT FROM AUTHOR]