Search

Comparative diagnostic test accuracy studies assess and compare the accuracy of 2 or more tests in the same study. Although these studies have the potential to yield reliable evidence regarding comparative accuracy, shortcomings in the design, conduct, and analysis may bias their results. The currently recommended quality assessment tool for diagnostic test accuracy studies, QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2), is not designed for the assessment of test comparisons. The QUADAS-C (Quality Assessment of Diagnostic Accuracy Studies-Comparative) tool was developed as an extension of QUADAS-2 to assess the risk of bias in comparative diagnostic test accuracy studies. Through a 4-round Delphi study involving 24 international experts in test evaluation and a face-to-face consensus meeting, an initial version of the tool was developed that was revised and finalized following a pilot study among potential users. The QUADAS-C tool retains the same 4-domain structure of QUADAS-2 (Patient Selection, Index Test, Reference Standard, and Flow and Timing) and comprises additional questions to each QUADAS-2 domain. A risk-of-bias judgment for comparative accuracy requires a risk-of-bias judgment for the accuracy of each test (resulting from QUADAS-2) and additional criteria specific to test comparisons. Examples of such additional criteria include whether participants either received all index tests or were randomly assigned to index tests, and whether index tests were interpreted with blinding to the results of other index tests. The QUADAS-C tool will be useful for systematic reviews of diagnostic test accuracy addressing comparative questions. Furthermore, researchers may use this tool to identify and avoid risk of bias when designing a comparative diagnostic test accuracy study.

Clear and informative reporting in titles and abstracts is essential to help readers and reviewers identify potentially relevant studies and decide whether to read the full text. Although the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) statement provides general recommendations for reporting titles and abstracts, more detailed guidance seems to be desirable. The authors present TRIPOD for Abstracts, a checklist and corresponding guidance for reporting prediction model studies in abstracts. A list of 32 potentially relevant items was the starting point for a modified Delphi procedure involving 110 experts, of whom 71 (65%) participated in the web-based survey. After 2 Delphi rounds, the experts agreed on 21 items as being essential to report in abstracts of prediction model studies. This number was reduced by merging some of the items. In a third round, participants provided feedback on a draft version of TRIPOD for Abstracts. The final checklist contains 12 items and applies to journal and conference abstracts that describe the development or external validation of a diagnostic or prognostic prediction model, or the added value of predictors to an existing model, regardless of the clinical domain or statistical approach used.

Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.

Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.

Background: The HEART (History, Electrocardiogram, Age, Risk factors, and initial Troponin) score is an easy-to-apply instrument to stratify patients with chest pain according to their short-term risk for major adverse cardiac events (MACEs), but its effect on daily practice is unknown., Objective: To measure the effect of use of the HEART score on patient outcomes and use of health care resources., Design: Stepped-wedge, cluster randomized trial. (ClinicalTrials.gov: NCT01756846)., Setting: Emergency departments in 9 Dutch hospitals., Patients: Unselected patients with chest pain presenting at emergency departments in 2013 and 2014., Intervention: All hospitals started with usual care. Every 6 weeks, 1 hospital was randomly assigned to switch to "HEART care," during which physicians calculated the HEART score to guide patient management., Measurements: For safety, a noninferiority margin of a 3.0% absolute increase in MACEs within 6 weeks was set. Other outcomes included use of health care resources, quality of life, and cost-effectiveness., Results: A total of 3648 patients were included (1827 receiving usual care and 1821 receiving HEART care). Six-week incidence of MACEs during HEART care was 1.3% lower than during usual care (upper limit of the 1-sided 95% CI, 2.1% [within the noninferiority margin of 3.0%]). In low-risk patients, incidence of MACEs was 2.0% (95% CI, 1.2% to 3.3%). No statistically significant differences in early discharge, readmissions, recurrent emergency department visits, outpatient visits, or visits to general practitioners were observed., Limitation: Physicians were hesitant to refrain from admission and diagnostic tests in patients classified as low risk by the HEART score., Conclusion: Using the HEART score during initial assessment of patients with chest pain is safe, but the effect on health care resources is limited, possibly due to nonadherence to management recommendations., Primary Funding Source: Netherlands Organisation for Health Research and Development.

Prediction models are developed to aid health care providers in estimating the probability or risk that a specific disease or condition is present (diagnostic models) or that a specific event will occur in the future (prognostic models), to inform their decision making. However, the overwhelming evidence shows that the quality of reporting of prediction model studies is poor. Only with full and clear reporting of information on all aspects of a prediction model can risk of bias and potential usefulness of prediction models be adequately assessed. The Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Initiative developed a set of recommendations for the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. This article describes how the TRIPOD Statement was developed. An extensive list of items based on a review of the literature was created, which was reduced after a Web-based survey and revised during a 3-day meeting in June 2011 with methodologists, health care professionals, and journal editors. The list was refined during several meetings of the steering group and in e-mail discussions with the wider group of TRIPOD contributors. The resulting TRIPOD Statement is a checklist of 22 items, deemed essential for transparent reporting of a prediction model study. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. The TRIPOD Statement is best used in conjunction with the TRIPOD explanation and elaboration document. To aid the editorial process and readers of prediction model studies, it is recommended that authors include a completed checklist in their submission (also available at www.tripod-statement.org).

The TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) Statement includes a 22-item checklist, which aims to improve the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. This explanation and elaboration document describes the rationale; clarifies the meaning of each item; and discusses why transparent reporting is important, with a view to assessing risk of bias and clinical usefulness of the prediction model. Each checklist item of the TRIPOD Statement is explained in detail and accompanied by published examples of good reporting. The document also provides a valuable reference of issues to consider when designing, conducting, and analyzing prediction model studies. To aid the editorial process and help peer reviewers and, ultimately, readers and systematic reviewers of prediction model studies, it is recommended that authors include a completed checklist in their submission. The TRIPOD checklist can also be downloaded from www.tripod-statement.org.

A universal challenge in studies that quantify the accuracy of diagnostic tests is establishing whether each participant has the disease of interest. Ideally, the same preferred reference standard would be used for all participants; however, for practical or ethical reasons, alternative reference standards that are often less accurate are frequently used instead. The use of different reference standards across participants in a single study is known as differential verification.Differential verification can cause severely biased accuracy estimates of the test or model being studied. Many variations of differential verification exist, but not all introduce the same risk of bias. A risk-of-bias assessment requires detailed information about which participants receive which reference standards and an estimate of the accuracy of the alternative reference standard. This article classifies types of differential verification and explores how they can lead to bias. It also provides guidance on how to report results and assess the risk of bias when differential verification occurs and highlights potential ways to correct for the bias.

In 2003, the QUADAS tool for systematic reviews of diagnostic accuracy studies was developed. Experience, anecdotal reports, and feedback suggested areas for improvement; therefore, QUADAS-2 was developed. This tool comprises 4 domains: patient selection, index test, reference standard, and flow and timing. Each domain is assessed in terms of risk of bias, and the first 3 domains are also assessed in terms of concerns regarding applicability. Signalling questions are included to help judge risk of bias. The QUADAS-2 tool is applied in 4 phases: summarize the review question, tailor the tool and produce review-specific guidance, construct a flow diagram for the primary study, and judge bias and applicability. This tool will allow for more transparent rating of bias and applicability of primary diagnostic accuracy studies.

Background: Clinical probability assessment is combined with d-dimer testing to exclude pulmonary embolism (PE)., Purpose: To compare the test characteristics of gestalt (a physician's unstructured estimate) and clinical decision rules for evaluating adults with suspected PE and assess the failure rate of gestalt and rules when used in combination with d-dimer testing., Data Sources: Articles in MEDLINE and EMBASE in English, French, German, Italian, Spanish, or Dutch that were published between 1966 and June 2011., Study Selection: 3 reviewers, working in pairs, selected prospective studies in consecutive patients suspected of having PE. Studies had to estimate the probability of PE by using gestalt or a decision rule and verify the diagnosis by using an appropriate reference standard., Data Extraction: Data on study characteristics, test performance, and prevalence were extracted. Reviewers constructed 2 × 2 tables and assessed the methodological quality of the studies., Data Synthesis: 52 studies, comprising 55 268 patients, were selected. Meta-analysis was performed on studies that used gestalt (15 studies; sensitivity, 0.85; specificity, 0.51), the Wells rule with a cutoff value less than 2 (19 studies; sensitivity, 0.84; specificity, 0.58) or 4 or less (11 studies; sensitivity, 0.60; specificity, 0.80), the Geneva rule (5 studies; sensitivity, 0.84; specificity, 0.50), and the revised Geneva rule (4 studies; sensitivity, 0.91; specificity, 0.37). An increased prevalence of PE was associated with higher sensitivity and lower specificity. Combining a decision rule or gestalt with d-dimer testing seemed safe for all strategies, except when the less-sensitive Wells rule (cutoff value ≤4) was combined with less-sensitive qualitative d-dimer testing., Limitations: Studies had substantial heterogeneity due to prevalence of PE and differences in threshold. Many studies (63%) had potential bias due to differential disease verification., Conclusion: Clinical decision rules and gestalt can safely exclude PE when combined with sensitive d-dimer testing. The authors recommend standardized rules because gestalt has lower specificity, but the choice of a particular rule and d-dimer test depend on both prevalence and setting.

Background: Studies of diagnostic accuracy are subject to different sources of bias and variation than studies that evaluate the effectiveness of an intervention. Little is known about the effects of these sources of bias and variation., Purpose: To summarize the evidence on factors that can lead to bias or variation in the results of diagnostic accuracy studies., Data Sources: MEDLINE, EMBASE, and BIOSIS, and the methodologic databases of the Centre for Reviews and Dissemination and the Cochrane Collaboration. Methodologic experts in diagnostic tests were contacted., Study Selection: Studies that investigated the effects of bias and variation on measures of test performance were eligible for inclusion, which was assessed by one reviewer and checked by a second reviewer. Discrepancies were resolved through discussion., Data Extraction: Data extraction was conducted by one reviewer and checked by a second reviewer., Data Synthesis: The best-documented effects of bias and variation were found for demographic features, disease prevalence and severity, partial verification bias, clinical review bias, and observer and instrument variation. For other sources, such as distorted selection of participants, absent or inappropriate reference standard, differential verification bias, and review bias, the amount of evidence was limited. Evidence was lacking for other features, including incorporation bias, treatment paradox, arbitrary choice of threshold value, and dropouts., Conclusions: Many issues in the design and conduct of diagnostic accuracy studies can lead to bias or variation; however, the empirical evidence about the size and effect of these issues is limited.

The quality of reporting of studies of diagnostic accuracy is less than optimal. Complete and accurate reporting is necessary to enable readers to assess the potential for bias in the study and to evaluate the generalizability of the results. A group of scientists and editors has developed the STARD (Standards for Reporting of Diagnostic Accuracy) statement to improve the reporting the quality of reporting of studies of diagnostic accuracy. The statement consists of a checklist of 25 items and flow diagram that authors can use to ensure that all relevant information is present. This explanatory document aims to facilitate the use, understanding, and dissemination of the checklist. The document contains a clarification of the meaning, rationale, and optimal use of each item on the checklist, as well as a short summary of the available evidence on bias and applicability. The STARD statement, checklist, flowchart, and this explanation and elaboration document should be useful resources to improve reporting of diagnostic accuracy studies. Complete and informative reporting can only lead to better decisions in health care.

Background: To comprehend the results of diagnostic accuracy studies, readers must understand the design, conduct, analysis, and results of such studies. That goal can be achieved only through complete transparency from authors., Objective: To improve the accuracy and completeness of reporting of studies of diagnostic accuracy in order to allow readers to assess the potential for bias in the study and to evaluate its generalizability., Methods: The Standards for Reporting of Diagnostic Accuracy (STARD) steering committee searched the literature to identify publications on the appropriate conduct and reporting of diagnostic studies and extracted potential items into an extensive list. Researchers, editors, methodologists and statisticians, and members of professional organizations shortened this list during a 2-day consensus meeting with the goal of developing a checklist and a generic flow diagram for studies of diagnostic accuracy., Results: The search for published guidelines on diagnostic research yielded 33 previously published checklists, from which we extracted a list of 75 potential items. The consensus meeting shortened the list to 25 items, using evidence on bias whenever available. A prototypical flow diagram provides information about the method of patient recruitment, the order of test execution, and the numbers of patients undergoing the test under evaluation, the reference standard, or both., Conclusions: Evaluation of research depends on complete and accurate reporting. If medical journals adopt the checklist and the flow diagram, the quality of reporting of studies of diagnostic accuracy should improve to the advantage of the clinicians, researchers, reviewers, journals, and the public.