Search

Background Randomised controlled trials are designed to assess the superiority, equivalence or non-inferiority of a new health technology, but which trial design should be used is not always obvious in practice. In particular, when using equivalence or non-inferiority designs, multiple outcomes of interest may be important for the success of a trial, despite the fact that usually only a single primary outcome is used to design the trial. Benefit–risk methods are used in the regulatory clinical trial setting to assess multiple outcomes and consider the trade-off of the benefits against the risks, but are not regularly implemented in publicly funded trials. Objectives The aim of the project is to aid the design of clinical trials with multiple outcomes of interest by defining when each trial design is appropriate to use and identifying when to use benefit–risk methods to assess outcome trade-offs (qualitatively or quantitatively) in a publicly funded trial setting. Methods A range of methods was used to elicit expert opinion to answer the project objectives, including a web-based survey of relevant researchers, a rapid review of current literature and a 2-day consensus workshop of experts (in 2019). Results We created a list of 19 factors to aid researchers in selecting the most appropriate trial design, containing the following overarching sections: population, intervention, comparator, outcomes, feasibility and perspectives. Six key reasons that indicate a benefit–risk method should be considered within a trial were identified: (1) when the success of the trial depends on more than one outcome; (2) when important outcomes within the trial are in competing directions (i.e. a health technology is better for one outcome, but worse for another); (3) to allow patient preferences to be included and directly influence trial results; (4) to provide transparency on subjective recommendations from a trial; (5) to provide consistency in the approach to presenting results from a trial; and (6) to synthesise multiple outcomes into a single metric. Further information was provided to support the use of benefit–risk methods in appropriate circumstances, including the following: methods identified from the review were collated into different groupings and described to aid the selection of a method; potential implementation of methods throughout the trial process were provided and discussed (with examples); and general considerations were described for those using benefit–risk methods. Finally, a checklist of five pieces of information that should be present when reporting benefit–risk methods was defined, with two additional items specifically for reporting the results. Conclusions These recommendations will assist research teams in selecting which trial design to use and deciding whether or not a benefit–risk method could be included to ensure research questions are answered appropriately. Additional information is provided to support consistent use and clear reporting of benefit–risk methods in the future. The recommendations can also be used by funding committees to confirm that appropriate considerations of the trial design have been made. Limitations This research was limited in scope and should be considered in conjunction with other trial design methodologies to assess appropriateness. In addition, further research is needed to provide concrete information about which benefit–risk methods are best to use in publicly funded trials, along with recommendations that are specific to each method. Study registration The rapid review is registered as PROSPERO CRD42019144882. Funding Funded by the Medical Research Council UK and the National Institute for Health and Care Research as part of the Medical Research Council–National Institute for Health and Care Research Methodology Research programme. Plain language summary Randomised controlled trials are considered the best way to gather evidence about potential NHS treatments. They can be designed from different perspectives depending whether the aim is to show that a new treatment is better than, equal to or no worse than the current best available treatment. The selection of this design relates to the single most important outcome; however, often multiple outcomes can be affected by a treatment. For example, a new treatment may improve disease management but increase side effects. Patients want a treatment to work but not at the price of poor quality of life; therefore, a trade-off must be made, and the recommended treatment depends on this trade-off. Benefit–risk methods can assess the trade-off between multiple outcomes and can include patient preference. These methods could improve the way that decisions are made about treatments in the NHS, but there is currently limited research about the use of these methods in publicly funded trials. The aim of this report is to improve the design of clinical trials by helping researchers to select the most appropriate trial design and to decide when to include a benefit–risk method. The recommendations were created using the opinions of experts within the field and consisted of a survey, review of the literature and a workshop. The project created a list of 19 factors that can assist researchers to select the most appropriate trial design. Furthermore, six key areas were identified in which researchers may consider including a benefit–risk method within a trial. Finally, if a benefit–risk assessment is being used, a checklist of items has been created that identifies the information important to include in reports. This report is, however, limited in its applicability and further research should extend this work, as well as provide more detail on individual methods that are available. Scientific summary Background Randomised controlled trials (RCTs) are the gold standard of health technology evaluation. They can be designed to assess the objective of superiority, equivalence or non-inferiority; the objective is determined by the selected research question and primary outcome. However, in practice, selecting the most appropriate outcome measure and, subsequently, trial design can be difficult. In addition, some trials may have more than one outcome of importance to consider; this is particularly the case when using equivalence or non-inferiority designs, as a secondary superiority outcome is often important. This multidimensionality is not considered if the focus of a trial’s success is a single primary outcome. Benefit–risk (B–R) methods are used in the clinical trial regulatory setting to assess multiple outcomes and consider the trade-off of the benefits against the risks, but they are not regularly implemented in publicly funded UK trials. Objectives This guidance document aims to fill a knowledge gap, with a focus on publicly funded clinical trials throughout. This is undertaken by first providing recommendations on the most appropriate trial design to select and then identifying when a B–R method may be used within a trial to assess outcome trade-offs, whether this be qualitatively or quantitatively. Methods In this project, three key methodologies were used across three work packages (WPs) to elicit expert opinion on trial design and B–R methods. These were: WP1, a web-based survey of relevant researchers in the area. The survey was sent to researchers through appropriate mailing lists, as well as through known contacts and networks, for completion. WP2, a rapid review of current literature. The review built on previous reviews to find available B–R methods, but, in addition to previous work, we retrieved details on which of these could be implemented in publicly funded studies, as well as details on each method’s strengths and weaknesses. WP3, a 2-day expert consensus workshop. Results from the survey and the review were presented to a group of experts (n = 15) who had been contacted because of their knowledge of and interest in the area. The nominal group technique was used to select items for inclusion in checklists. Open discussions, supported by the presented findings, were used to gain opinions on reasons for including B–R methods within a trial. Open discussions were thematically analysed to identify the key points discussed and support the results of the checklists. Results To aid researchers in selecting an appropriate trial design, a list of 19 factors to consider was created. This list relates to six different sections: population, intervention, comparator, outcomes, feasibility and perspectives. Each factor is described in detail and the list of factors is used in conjunction with examples of trial designs to explain why a particular design might be chosen. Once the appropriate trial design has been selected, considering a B–R method may be justified if one of six key reasons is present: The success of the trial depends on more than one outcome. Important outcomes within the trial are in competing directions (i.e. a health technology is anticipated to be better on one outcome but worse on another). To allow patient preferences to be included and directly influence trial results. To provide transparency on subjective recommendations from a trial. To provide consistency in the approach to presenting results from a trial. To synthesise multiple outcomes into a single metric. In particular, the first reason (considering multiple outcomes) is often applicable to equivalence and non-inferiority trials, in which success also depends on the superiority of a secondary outcome; a description of this was flagged as important in the consensus workshop. Owing to the range of B–R methods available, difficulties arise when discussing a globally recommended B–R assessment. To aid with this, the 92 methods that were identified from the literature review as potentially suitable for use in publicly funded RCTs were categorised into seven groups. The first group was an overarching framework that provided structure for a B–R assessment from start to finish. The most basic group, the second group, was the use of a narrative summary of the benefits and risks; although this is not an official method, it was considered important to provide such a summary at the end of a trial. The next two groups comprised a summary table with a specific structure to contain all relevant information, before moving on to methods that can quantitatively trade off multiple outcomes. The final three groups related to preference elicitation from stakeholders, uncertainty estimation and visualisations. Importantly, these groupings show that quantitative methods are not always necessary or appropriate. However, this should not negate the need for a summary or narrative discussion of subjective interpretations. As it could be argued that favourable and unfavourable outcomes are present in all RCTs, a narrative summary of the benefits and risks would add value to the reports of all publicly funded trials. It is noted that some methods require additional data to be collected, so the need for these added resources and expenses should be considered on a case-by-case basis. To provide some clarity on this, the methods groups were linked with different stages of the RCT, from design through to conclusion and dissemination. Group discussions identified the difficulty of additional work during the trial design stage, which is something that this breakdown of the methods at different stages hopes to address. Finally, when a B–R method has been deemed important to use, whether that be a narrative summary or a quantitative trade-off method, the reporting should be transparent and consistent. A checklist of items to include when reporting the trial design in applications or protocols was created, as was a separate checklist for reporting the trial findings. There are five pieces of information that are recommended when reporting the trial design: a heading of ‘benefit–risk’ explicit use of the term ‘benefit–risk’ a plan for B–R assessment the anticipated benefits and risks discussion of the B–R balance with patient representatives. Two further items are recommended when reporting the results of the trial: a summary table of benefits and risks reporting of quality-adjusted life-years in terms of B–R. Conclusions The advice and recommendations provided in this report aim to improve the selection of a trial design, understanding of B–R methods and when it is appropriate to include them and how to effectively report the use of the methods. This guidance is aimed at researchers designing publicly funded RCTs, such as those by the National Institute for Health and Care Research or the Medical Research Council, as well as those on the funding panels. Using these recommendations will improve the design of RCTs, as well as the clarity and consistency of reporting the benefits and risks associated with the health technologies being evaluated. Further research should focus on providing more detail on each of the different B–R methods and recommendations on how they can be fully integrated in publicly funded clinical trials. Practical barriers to this implementation could be assessed to ensure that the information presented in this report can be used most effectively. Study registration This study is registered as PROSPERO CRD42019144882. Funding Funded by the Medical Research Council UK and the National Institute for Health and Care Research as part of the Medical Research Council–National Institute for Health and Care Research Methodology Research programme.

Background: The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the ‘target difference’ and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups. Objective: The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives. Methods: The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document. Results: Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered. Conclusions: Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items. Funding: Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC–National Institute for Health Research Methodology Research programme.

Background: Asthma episodes and deaths are known to be seasonal. A number of reports have shown peaks in asthma episodes in school-aged children associated with the return to school following the summer vacation. A fall in prescription collection in the month of August has been observed, and was associated with an increase in the number of unscheduled contacts after the return to school in September. Objective: The primary objective of the study was to assess whether or not a NHS-delivered public health intervention reduces the September peak in unscheduled medical contacts. Design: Cluster randomised trial, with the unit of randomisation being 142 NHS general practices, and trial-based economic evaluation. Setting: Primary care. Intervention: A letter sent (n = 70 practices) in July from their general practitioner (GP) to parents/carers of school-aged children with asthma to remind them of the importance of taking their medication, and to ensure that they have sufficient medication prior to the start of the new school year in September. The control group received usual care. Main outcome measures: The primary outcome measure was the proportion of children aged 5–16 years who had an unscheduled medical contact in September 2013. Supporting end points included the proportion of children who collected prescriptions in August 2013 and unscheduled contacts through the following 12 months. Economic end points were quality-adjusted life-years (QALYs) gained and costs from an NHS and Personal Social Services perspective. Results: There is no evidence of effect in terms of unscheduled contacts in September. Among children aged 5–16 years, the odds ratio (OR) was 1.09 [95% confidence interval (CI) 0.96 to 1.25] against the intervention. The intervention did increase the proportion of children collecting a prescription in August (OR 1.43, 95% CI 1.24 to 1.64) as well as scheduled contacts in the same month (OR 1.13, 95% CI 0.84 to 1.52). For the wider time intervals (September–December 2013 and September–August 2014), there is weak evidence of the intervention reducing unscheduled contacts. The intervention did not reduce unscheduled care in September, although it succeeded in increasing the proportion of children collecting prescriptions in August as well as having scheduled contacts in the same month. These unscheduled contacts in September could be a result of the intervention, as GPs may have wanted to see patients before issuing a prescription. The economic analysis estimated a high probability that the intervention was cost-saving, for baseline-adjusted costs, across both base-case and sensitivity analyses. There was no increase in QALYs. Limitation: The use of routine data led to uncertainty in the coding of medical contacts. The uncertainty was mitigated by advice from a GP adjudication panel. Conclusions: The intervention did not reduce unscheduled care in September, although it succeeded in increasing the proportion of children both collecting prescriptions and having scheduled contacts in August. After September there is weak evidence in favour of the intervention. The intervention had a favourable impact on costs but did not demonstrate any impact on QALYs. The results of the trial indicate that further work is required on assessing and understanding adherence, both in terms of using routine data to make quantitative assessments, and through additional qualitative interviews with key stakeholders such as practice nurses, GPs and a wider group of children with asthma. Trial registration: Current Controlled Trials ISRCTN03000938. Funding details: This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 20, No. 93. See the HTA programme website for further project information.