Search

Abstract Adaptive designs are a class of methods for improving efficiency and patient benefit of clinical trials. Although their use has increased in recent years, research suggests they are not used in many situations where they have potential to bring benefit. One barrier to their more widespread use is a lack of understanding about how the choice to use an adaptive design, rather than a traditional design, affects resources (staff and non-staff) required to set-up, conduct and report a trial. The Costing Adaptive Trials project investigated this issue using quantitative and qualitative research amongst UK Clinical Trials Units. Here, we present guidance that is informed by our research, on considering the appropriate resourcing of adaptive trials. We outline a five-step process to estimate the resources required and provide an accompanying costing tool. The process involves understanding the tasks required to undertake a trial, and how the adaptive design affects them. We identify barriers in the publicly funded landscape and provide recommendations to trial funders that would address them. Although our guidance and recommendations are most relevant to UK non-commercial trials, many aspects are relevant more widely.

Background Paediatric trials must contend with many challenges that adult trials face but often bring additional obstacles. Decentralised trials, where some or all trial methods occur away from a centralised location, are a promising strategy to help meet these challenges. This scoping review aims to (a) identify what methods and tools have been used to create and conduct entirely online-decentralised trials with children and (b) determine the gaps in the knowledge in this field. This review will describe the methods used in these trials to identify their facilitators and the gaps in the knowledge. Methods The methods were informed by guidance from the Joanna Briggs Institute and the PRISMA extension for scoping reviews. We systematically searched MEDLINE, CENTRAL, CINAHL, and Embase databases, trial registries, pre-print servers, and the internet. We included randomised and quasi-randomised trials conducted entirely online with participants under 18 published in English. A risk of bias assessment was completed for all included studies. Results Twenty-one trials met our inclusion criteria. The average age of participants was 14.6 years. Social media was the most common method of online recruitment. Most trials employed an external host website to store and protect their data. Duration of trials ranged from single-session interventions up to ten weeks. Fourteen trials compensated participants. Eight trials involved children in their trial design process; none reported compensation for this. Most trials had a low risk of bias in “random sequence generation”, “selective reporting”, and “other”. Most trials had a high risk of bias in “blinding participants and personnel”, “blinding of outcome assessment”, and “incomplete outcome data”. “Allocation concealment” was unclear in most studies. Conclusions There was a lack of transparent reporting of the recruitment, randomisation, and retention methods used in many of the trials included in this review. Patient and public involvement (PPI) was not common, and the compensation of PPI partners was not reported in any study. Consent methods and protection against fraudulent entries to trials were creative and thoroughly discussed by some trials and not addressed by others. More work and thorough reporting of how these trials are conducted is needed to increase their reproducibility and quality. Ethics and dissemination Ethical approval was not necessary since all data sources used are publicly available.

BackgroundPaediatric trials must contend with many challenges that adult trials face but often bring additional obstacles. Decentralised trials, where some or all trial methods occur away from a centralised location, are a promising strategy to help meet these challenges. This scoping review aims to (a) identify what methods and tools have been used to create and conduct entirely online-decentralised trials with children and (b) determine the gaps in the knowledge in this field. This review will describe the methods used in these trials to identify their facilitators and the gaps in the knowledge.MethodsThe methods were informed by guidance from the Joanna Briggs Institute and the PRISMA extension for scoping reviews. We systematically searched MEDLINE, CENTRAL, CINAHL, and Embase databases, trial registries, pre-print servers, and the internet. We included randomised and quasi-randomised trials conducted entirely online with participants under 18 published in English. A risk of bias assessment was completed for all included studies.ResultsTwenty-one trials met our inclusion criteria. The average age of participants was 14.6 years. Social media was the most common method of online recruitment. Most trials employed an external host website to store and protect their data. Duration of trials ranged from single-session interventions up to ten weeks. Fourteen trials compensated participants. Eight trials involved children in their trial design process; none reported compensation for this. Most trials had a low risk of bias in "random sequence generation", "selective reporting", and "other". Most trials had a high risk of bias in "blinding participants and personnel", "blinding of outcome assessment", and "incomplete outcome data". "Allocation concealment" was unclear in most studies.ConclusionsThere was a lack of transparent reporting of the recruitment, randomisation, and retention methods used in many of the trials included in this review. Patient and public involvement (PPI) was not common, and the compensation of PPI partners was not reported in any study. Consent methods and protection against fraudulent entries to trials were creative and thoroughly discussed by some trials and not addressed by others. More work and thorough reporting of how these trials are conducted is needed to increase their reproducibility and quality.Ethics and disseminationEthical approval was not necessary since all data sources used are publicly available.

Abstract Background Adaptive designs offer great promise in improving the efficiency and patient-benefit of clinical trials. An important barrier to further increased use is a lack of understanding about which additional resources are required to conduct a high-quality adaptive clinical trial, compared to a traditional fixed design. The Costing Adaptive Trials (CAT) project investigated which additional resources may be required to support adaptive trials. Methods We conducted a mock costing exercise amongst seven Clinical Trials Units (CTUs) in the UK. Five scenarios were developed, derived from funded clinical trials, where a non-adaptive version and an adaptive version were described. Each scenario represented a different type of adaptive design. CTU staff were asked to provide the costs and staff time they estimated would be needed to support the trial, categorised into specified areas (e.g. statistics, data management, trial management). This was calculated separately for the non-adaptive and adaptive version of the trial, allowing paired comparisons. Interviews with 10 CTU staff who had completed the costing exercise were conducted by qualitative researchers to explore reasons for similarities and differences. Results Estimated resources associated with conducting an adaptive trial were always (moderately) higher than for the non-adaptive equivalent. The median increase was between 2 and 4% for all scenarios, except for sample size re-estimation which was 26.5% (as the adaptive design could lead to a lengthened study period). The highest increase was for statistical staff, with lower increases for data management and trial management staff. The percentage increase in resources varied across different CTUs. The interviews identified possible explanations for differences, including (1) experience in adaptive trials, (2) the complexity of the non-adaptive and adaptive design, and (3) the extent of non-trial specific core infrastructure funding the CTU had. Conclusions This work sheds light on additional resources required to adequately support a high-quality adaptive trial. The percentage increase in costs for supporting an adaptive trial was generally modest and should not be a barrier to adaptive designs being cost-effective to use in practice. Informed by the results of this research, guidance for investigators and funders will be developed on appropriately resourcing adaptive trials.

Abstract Introduction Adaptive designs offer a flexible approach, allowing changes to a trial based on examinations of the data as it progresses. Adaptive clinical trials are becoming a popular choice, as the prudent use of finite research budgets and accurate decision-making are priorities for healthcare providers around the world. The methods of health economics, which aim to maximise the health gained for money spent, could be incorporated into the design and analysis of adaptive clinical trials to make them more efficient. We aimed to understand the perspectives of stakeholders in health technology assessments to inform recommendations for the use of health economics in adaptive clinical trials. Methods A qualitative study explored the attitudes of key stakeholders—including researchers, decision-makers and members of the public—towards the use of health economics in the design and analysis of adaptive clinical trials. Data were collected using interviews and focus groups (29 participants). A framework analysis was used to identify themes in the transcripts. Results It was considered that answering the clinical research question should be the priority in a clinical trial, notwithstanding the importance of cost-effectiveness for decision-making. Concerns raised by participants included handling the volatile nature of cost data at interim analyses; implementing this approach in global trials; resourcing adaptive trials which are designed and adapted based on health economic outcomes; and training stakeholders in these methods so that they can be implemented and appropriately interpreted. Conclusion The use of health economics in the design and analysis of adaptive clinical trials has the potential to increase the efficiency of health technology assessments worldwide. Recommendations are made concerning the development of methods allowing the use of health economics in adaptive clinical trials, and suggestions are given to facilitate their implementation in practice.

Objectives To identify how frequently patient-reported outcomes (PROs) are used as primary and/or secondary outcomes in randomised controlled trials (RCTs) and to summarise what statistical methods are used for the analysis of PROs.Design Comprehensive review.Setting RCTs funded and published by the United Kingdom’s (UK) National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme.Data sources and eligibility HTA reports of RCTs published between January 1997 and December 2020 were reviewed.Data extraction Information relating to PRO use and analysis methods was extracted.Primary and secondary outcome measures The frequency of using PROs as primary and/or secondary outcomes; statistical methods that were used for the analysis of PROs as primary outcomes.Results In this review, 37.6% (114/303) of trials used PROs as primary outcomes, and 82.8% (251/303) of trials used PROs as secondary outcomes from 303 NIHR HTA reports of RCTs. In the 114 RCTs where the PRO was the primary outcome, the most used PRO was the Short-Form 36 (8/114); the most popular methods for multivariable analysis were linear mixed model (45/114), linear regression (29/114) and analysis of covariance (13/114); logistic regression was applied for binary and ordinal outcomes in 14/114 trials; and the repeated measures analysis was used in 39/114 trials.Conclusion The majority of trials used PROs as primary and/or secondary outcomes. Conventional methods such as linear regression are widely used, despite the potential violation of their assumptions. In recent years, there is an increasing trend of using complex models (eg, with mixed effects). Statistical methods developed to address these violations when analysing PROs, such as beta-binomial regression, are not routinely used in practice. Future research will focus on evaluating available statistical methods for the analysis of PROs.

Background: Licensed ranibizumab (0.5 mg/0.05 ml Lucentis®; Novartis International AG, Basel, Switzerland) and aflibercept (2 mg/0.05 ml Eylea®; Bayer AG, Leverkusen, Germany) and unlicensed bevacizumab (1.25 mg/0.05 ml Avastin®; F. Hoffmann-La Roche AG, Basel, Switzerland) are used to treat macula oedema due to central retinal vein occlusion, but their relative clinical effectiveness, cost-effectiveness and impact on the UK NHS and Personal Social Services have never been directly compared over the typical disease treatment period. Objective: The objective was to compare the clinical effectiveness and cost-effectiveness of three intravitreal antivascular endothelial growth factor agents for the management of macula oedema due to central retinal vein occlusion. Design: This was a three-arm, double-masked, randomised controlled non-inferiority trial. Setting: The trial was set in 44 UK NHS ophthalmology departments, between 2014 and 2018. Participants: A total of 463 patients with visual impairment due to macula oedema secondary to central retinal vein occlusion were included in the trial. Interventions: The participants were treated with repeated intravitreal injections of ranibizumab (n = 155), aflibercept (n = 154) or bevacizumab (n = 154). Main outcome measures: The primary outcome was an increase in the best corrected visual acuity letter score from baseline to 100 weeks in the trial eye. The null hypothesis that aflibercept and bevacizumab are each inferior to ranibizumab was tested with a non-inferiority margin of –5 visual acuity letters over 100 weeks. Secondary outcomes included additional visual acuity, and imaging outcomes, Visual Function Questionnaire-25, EuroQol-5 Dimensions with and without a vision bolt-on, and drug side effects. Cost-effectiveness was estimated using treatment costs and Visual Function Questionnaire-Utility Index to measure quality-adjusted life-years. Results: The adjusted mean changes at 100 weeks in the best corrected visual acuity letter scores were as follows – ranibizumab, 12.5 letters (standard deviation 21.1 letters); aflibercept, 15.1 letters (standard deviation 18.7 letters); and bevacizumab, 9.8 letters (standard deviation 21.4 letters). Aflibercept was non-inferior to ranibizumab in the intention-to-treat population (adjusted mean best corrected visual acuity difference 2.23 letters, 95% confidence interval –2.17 to 6.63 letters; p = 0.0006), but not superior. The study was unable to demonstrate that bevacizumab was non-inferior to ranibizumab in the intention-to-treat population (adjusted mean best corrected visual acuity difference –1.73 letters, 95% confidence interval –6.12 to 2.67 letters; p = 0.071). A post hoc analysis was unable to demonstrate that bevacizumab was non-inferior to aflibercept in the intention-to-treat population (adjusted mean best corrected visual acuity difference was –3.96 letters, 95% confidence interval –8.34 to 0.42 letters; p = 0.32). All per-protocol population results were the same. Fewer injections were required with aflibercept (10.0) than with ranibizumab (11.8) (difference in means –1.8, 95% confidence interval –2.9 to –0.8). A post hoc analysis showed that more bevacizumab than aflibercept injections were required (difference in means 1.6, 95% confidence interval 0.5 to 2.7). There were no new safety concerns. The model- and trial-based cost-effectiveness analyses estimated that bevacizumab was the most cost-effective treatment at a threshold of £20,000–30,000 per quality-adjusted life-year. Limitations: The comparison of aflibercept and bevacizumab was a post hoc analysis. Conclusion: The study showed aflibercept to be non-inferior to ranibizumab. However, the possibility that bevacizumab is worse than ranibizumab and aflibercept by 5 visual acuity letters cannot be ruled out. Bevacizumab is an economically attractive treatment alternative and would lead to substantial cost savings to the NHS and other health-care systems. However, uncertainty about its relative effectiveness should be discussed comprehensively with patients, their representatives and funders before treatment is considered. Future work: To obtain extensive patient feedback and discuss with all stakeholders future bevacizumab NHS use. Trial registration: Current Controlled Trials ISRCTN13623634. Funding: This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 38. See the NIHR Journals Library website for further project information.

Abstract Background In individually randomised trials we might expect interventions delivered in groups or by care providers to result in clustering of outcomes for participants treated in the same group or by the same care provider. In partially nested randomised controlled trials (pnRCTs) this clustering only occurs in one trial arm, commonly the intervention arm. It is important to measure and account for between-cluster variability in trial design and analysis. We compare analysis approaches for pnRCTs with continuous outcomes, investigating the impact on statistical inference of cluster sizes, coding of the non-clustered arm, intracluster correlation coefficient (ICCs), and differential variance between intervention and control arm, and provide recommendations for analysis. Methods We performed a simulation study assessing the performance of six analysis approaches for a two-arm pnRCT with a continuous outcome. These include: linear regression model; fully clustered mixed-effects model with singleton clusters in control arm; fully clustered mixed-effects model with one large cluster in control arm; fully clustered mixed-effects model with pseudo clusters in control arm; partially nested homoscedastic mixed effects model, and partially nested heteroscedastic mixed effects model. We varied the cluster size, number of clusters, ICC, and individual variance between the two trial arms. Results All models provided unbiased intervention effect estimates. In the partially nested mixed-effects models, methods for classifying the non-clustered control arm had negligible impact. Failure to account for even small ICCs resulted in inflated Type I error rates and over-coverage of confidence intervals. Fully clustered mixed effects models provided poor control of the Type I error rates and biased ICC estimates. The heteroscedastic partially nested mixed-effects model maintained relatively good control of Type I error rates, unbiased ICC estimation, and did not noticeably reduce power even with homoscedastic individual variances across arms. Conclusions In general, we recommend the use of a heteroscedastic partially nested mixed-effects model, which models the clustering in only one arm, for continuous outcomes similar to those generated under the scenarios of our simulations study. However, with few clusters (3–6), small cluster sizes (5–10), and small ICC (≤0.05) this model underestimates Type I error rates and there is no optimal model.

Abstract Adaptive designs can make clinical trials more flexible by utilising results accumulating in the trial to modify the trial’s course in accordance with pre-specified rules. Trials with an adaptive design are often more efficient, informative and ethical than trials with a traditional fixed design since they often make better use of resources such as time and money, and might require fewer participants. Adaptive designs can be applied across all phases of clinical research, from early-phase dose escalation to confirmatory trials. The pace of the uptake of adaptive designs in clinical research, however, has remained well behind that of the statistical literature introducing new methods and highlighting their potential advantages. We speculate that one factor contributing to this is that the full range of adaptations available to trial designs, as well as their goals, advantages and limitations, remains unfamiliar to many parts of the clinical community. Additionally, the term adaptive design has been misleadingly used as an all-encompassing label to refer to certain methods that could be deemed controversial or that have been inadequately implemented. We believe that even if the planning and analysis of a trial is undertaken by an expert statistician, it is essential that the investigators understand the implications of using an adaptive design, for example, what the practical challenges are, what can (and cannot) be inferred from the results of such a trial, and how to report and communicate the results. This tutorial paper provides guidance on key aspects of adaptive designs that are relevant to clinical triallists. We explain the basic rationale behind adaptive designs, clarify ambiguous terminology and summarise the utility and pitfalls of adaptive designs. We discuss practical aspects around funding, ethical approval, treatment supply and communication with stakeholders and trial participants. Our focus, however, is on the interpretation and reporting of results from adaptive design trials, which we consider vital for anyone involved in medical research. We emphasise the general principles of transparency and reproducibility and suggest how best to put them into practice.

Abstract Background In an individually randomised controlled trial where the treatment is delivered by a health professional it seems likely that the effectiveness of the treatment, independent of any treatment effect, could depend on the skill, training or even enthusiasm of the health professional delivering it. This may then lead to a potential clustering of the outcomes for patients treated by the same health professional, but similar clustering may not occur in the control arm. Using four case studies, we aim to provide practical guidance and recommendations for the analysis of trials with some element of clustering in one arm. Methods Five approaches to the analysis of outcomes from an individually randomised controlled trial with clustering in one arm are identified in the literature. Some of these methods are applied to four case studies of completed randomised controlled trials with clustering in one arm with sample sizes ranging from 56 to 539. Results are obtained using the statistical packages R and Stata and summarised using a forest plot. Results The intra-cluster correlation coefficient (ICC) for each of the case studies was small (