Search

Objectives: The STAndards for Reporting studies of Diagnostic accuracy (STARD) for investigators and editors and the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) for reviewers and readers offer guidelines for the quality and reporting of test accuracy studies. These guidelines address and propose some solutions to two major threats to validity: spectrum bias and test review bias. Study Design and Setting: Using a clinical example, we demonstrate that these solutions fail and propose an alternative Solution that concomitantly addresses both Sources of bias. We also derive formulas that prove the generality of our arguments. Results: A logical extension of our ideas is to extend STARD item 23 by adding a requirement for multivariable statistical adjustment using information collected in QUADAS items 1, 2, and 12 and STARD items 3-5, 11, 15, and 18. Conclusion: We recommend reporting not only variation of diagnostic accuracy across subgroups (STARD item 23) but also the effects of the multivariable adjustments on test performance. We also suggest that the QUADAS be supplemented by an item addressing the appropriateness of statistical methods, in particular whether multivariable adjustments have been included in the analysis. (C) 2009 Elsevier Inc. All rights reserved

Objective Meta-analysis of studies of the accuracy of diagnostic tests currently uses a variety of methods. Statistically rigorous hierarchical models require expertise and sophisticated software. We assessed whether any of the simpler methods can in practice give adequately accurate and reliable results. Study Design and Setting We reviewed six methods for meta-analysis of diagnostic accuracy: four simple commonly used methods (simple pooling, separate random-effects meta-analyses of sensitivity and specificity, separate meta-analyses of positive and negative likelihood ratios, and the Littenberg–Moses summary receiver operating characteristic [ROC] curve) and two more statistically rigorous approaches using hierarchical models (bivariate random-effects meta-analysis and hierarchical summary ROC curve analysis). We applied the methods to data from a sample of eight systematic reviews chosen to illustrate a variety of patterns of results. Results In each meta-analysis, there was substantial heterogeneity between the results of different studies. Simple pooling of results gave misleading summary estimates of sensitivity and specificity in some meta-analyses, and the Littenberg–Moses method produced summary ROC curves that diverged from those produced by more rigorous methods in some situations. Conclusion The closely related hierarchical summary ROC curve or bivariate models should be used as the standard method for meta-analysis of diagnostic accuracy.

We read the comment of Prof. Bossuyt with great interest and are happy to respond. We fully agree that the STARD and QUADAS instrument serve different purposes (despite substantial overlap in the topics these instruments address). Nowhere in our article did we suggest the opposite. For an intelligent investigator who plans ahead, the STARD initiative serves as a recipe for design, execution, and reporting alike. Therefore, stating that STARD is solely a set of rules for reporting and not a ‘‘prescription of how a diagnostic study ought to be done,’’ seems to us an undervaluation of the STARD guideline and a somewhat semantic argument. Moreover, we learn that the occurrences we describe in our article are standard knowledge in epidemiology for many years. If that is truly so, we wonder why the developers of the STARD statement did not suitably incorporate them. As mentioned in our conclusions, the phenomenon we treat is much more important for the validity of the reported test parameters and completely different from the subgroup variability issue as discussed in STARD item 23 and discussed in Prof. Bossuyt’s comments. The main message of our article is the recommendation that researchers investigate the influence of diagnostic covariates and report this influence of the effect on the test parameters. These covariates could influence the test parameters in two ways: different distributions of the covariates in the study population would produce different test parameters and different levels of integrating the information of these covariates in the test result will also produce different test parameters. Prof. Bossuyt advocates the unconditional summary accuracy statistics as the expression of the probability of

Objectives To identify key principles of graph construction and to review guidance on graph construction provided by medical journals. Study Design and Setting Review of a prevailing model of graph comprehension and survey of authors' instructions for all 120 core medical journals as indexed by PubMed. Results Graph construction should meet three requirements for optimal graph comprehension: Data symbols should be clearly distinguishable to enable their visual detection. Second, the data should be arranged in such a way that estimation of values and important relationships in the data is facilitated, and third, there should be a full explanation of all graph elements and of how the data were obtained. Seven journals (5.8%) provided recommendations on graph construction. Six journals put emphasis primarily on the explanation of graph elements and, to a lesser extent, on the visual detection of data symbols. Recommendations of the seventh journal focused mostly on how to enhance estimation of important relationships. All journals presented single recommendations whose selection seemed unstructured and not based on key principles of graph construction. Conclusion Medical journals should order their instructions according to the three key aspects of graph construction.