Search

Background: The GetReal Trial Tool is a decision support tool to assess the impact of design choices on generalizability of clinical trials to routine clinical practice, while taking into account the risk of bias, precision, acceptability and operational feasibility. This study describes the validation of the GetReal Trial Tool., Methods: Twelve experts took part in the GetReal Trial tool validation using the protocols of 6 trials conducted with pragmatic elements. The tool entails 7 domains with a total of 43 questions. A pooled Kappa statistic (95% CI) using random effects model was estimated using Open Meta (analyst) software. The possible operational challenges were collated and discussed with the trialists that conducted the trials., Results: Agreement in the design choices made for the trial protocols was >50% for all the trials and all teams reached consensus during discussion. The pooled Kappa statistic (95% CI) was 0.236 (0.154-0.318). The GetReal Trial tool highlighted several operational challenges, of which almost half had been experienced previously by the trialists. Out of 25 additional operational challenges mentioned by the trialists, 76% were already highlighted by the tool. The tool was considered helpful to optimize trials right from the design stage., Conclusion: The GetReal Trial Tool helps to scrutinize the choice of study design in the light of Real World Evidence generation. The tool identifies most of the operational challenges experienced by trialists to date. The tool serves the intended purpose of facilitating discussion and understanding more pragmatic design choices and their implications., Competing Interests: Declaration of Competing Interest The authors declare no conflict of interest., (Copyright © 2022 The Authors. Published by Elsevier Inc. All rights reserved.)

Methodologies incorporating Real World Elements into clinical trial design (also called pragmatic trials) offer an attractive opportunity to assess the effect of a treatment strategy in routine care and as such guide decision making in practice. Uptake of these methods is slow for several reasons, including uncertainty about acceptability of trial results, lack of experience with the methodology and operational challenges. We developed the "GetReal Trial Tool," an easy-to-use online interface, which allows users to assess the impact of design choices on generalizability to routine clinical practice, while taking into account risk of bias, precision, acceptability and operational feasibility. The tool is grounded in the scientific literature combined with knowledge of experts from academia, pharmaceutical companies, HTA bodies, patient organizations, and regulators. The aim is to help researchers optimize trial design and facilitate translation of evidence from pragmatic trials to clinical practice. In this paper we describe the development, structure and application of the GetReal Trial Tool., (Copyright © 2021. Published by Elsevier Inc.)

Results from pragmatic trials should reflect the comparative treatment effects encountered in patients in real-life clinical practice to guide treatment decisions. Therefore, pragmatic trials should focus on outcomes that are relevant to patients, clinical practice, and treatment choices. This sixth article in the series (see Box) discusses different types of outcomes and their suitability for pragmatic trials, design choices for measuring these outcomes, and their implications and challenges. Measuring outcomes in pragmatic trials should not interfere with real-world clinical practice to ensure generalizability of trial results, and routinely collected outcomes should be prioritized. Typical outcomes include mortality, morbidity, functional status, well-being, and resource use. Surrogate endpoints are typically avoided as primary outcome. It is important to measure outcomes over a relevant time horizon and obtain valid and precise results. As pragmatic trials are often open label, a less subjective outcome can reduce bias. Methods that decrease bias or enhance precision of the results, such as standardization and blinding of outcome assessment, should be considered when a high risk of bias or high variability is expected. The selection of outcomes in pragmatic trials should be relevant for decision making and feasible in terms of executing the trial in the context of interest. Therefore, this should be discussed with all stakeholders as early as feasible to ensure the relevance of study results for decision making in clinical practice and the ability to perform the study., (Copyright © 2017 The Authors. Published by Elsevier Inc. All rights reserved.)

Objectives: To illustrate how pragmatic trial design elements or inserting explanatory trial elements in pragmatic trials affect validity, generalizability, precision, and operational feasibility., Study Design and Setting: From illustrative examples identified through the IMI Get Real Consortium, we selected randomized drug trials with a pragmatic design feature. We searched all publications on these trials for information on how pragmatic trial design features affect validity, generalizability, precision, or feasibility., Results: We present examples from the Salford lung study, International Suicide Prevention Trial, Sequenced Treatment Alternatives to Relieve Depression, and Cluster Randomized Usual care vs. Caduet Investigation Assessing Long-term-risk trial. These examples show that incorporating pragmatic trial design elements in trials may affect generalizability, precision and validity and may lead to operational challenges different from traditional explanatory trials. Inserting explanatory trial elements into pragmatic trials may also affect validity, generalizability, and operational feasibility, especially when these trial elements are incorporated in one arm of the trial only. Design choices that positively affect one of these domains (e.g., generalizability) may negatively affect others (e.g., feasibility)., Conclusion: Consequences of incorporating pragmatic or explanatory trial design elements in pragmatic trials should be explicitly considered and balanced for all relevant domains, including validity, generalizability, precision, and operational feasibility. Tools are needed to make these consequences more transparent., (Copyright © 2016 Elsevier Inc. All rights reserved.)