Search

Recent FDA guidance on adaptive clinical trial designs defines bias as "a systematic tendency for the estimate of treatment effect to deviate from its true value," and states that it is desirable to obtain and report estimates of treatment effects that reduce or remove this bias. The conventional end-of-trial point estimates of the treatment effects are prone to bias in many adaptive designs, because they do not take into account the potential and realized trial adaptations. While much of the methodological developments on adaptive designs have tended to focus on control of type I error rates and power considerations, in contrast the question of biased estimation has received relatively less attention. This article is the first in a two-part series that studies the issue of potential bias in point estimation for adaptive trials. Part I provides a comprehensive review of the methods to remove or reduce the potential bias in point estimation of treatment effects for adaptive designs, while part II illustrates how to implement these in practice and proposes a set of guidelines for trial statisticians. The methods reviewed in this article can be broadly classified into unbiased and bias-reduced estimation, and we also provide a classification of estimators by the type of adaptive design. We compare the proposed methods, highlight available software and code, and discuss potential methodological gaps in the literature.

Background We aimed to assess the cost effectiveness of intravitreal ranibizumab (Lucentis), aflibercept (Eylea) and bevacizumab (Avastin) for the treatment of macular oedema due to central retinal vein occlusion. Methods We calculated costs and quality-adjusted life-years from the UK National Health Service and Personal Social Services perspective. We performed a within-trial analysis using the efficacy, safety, resource use and health utility data from a randomised controlled trial (LEAVO) over 100 weeks. We built a discrete event simulation to model long-term outcomes. We estimated utilities using the Visual-Functioning Questionnaire-Utility Index, EQ-5D and EQ-5D with an additional vision question. We used standard UK costs sources for 2018/19 and a cost of £28 per bevacizumab injection. We discounted costs and quality-adjusted life-years at 3.5% annually. Results Bevacizumab was the least costly intervention followed by ranibizumab and aflibercept in both the within-trial analysis (bevacizumab: £6292, ranibizumab: £13,014, aflibercept: £14,328) and long-term model (bevacizumab: £18,353, ranibizumab: £30,226, aflibercept: £35,026). Although LEAVO did not demonstrate bevacizumab to be non-inferior for the visual acuity primary outcome, the three interventions generated similar quality-adjusted life-years in both analyses. Bevacizumab was always the most cost-effective intervention at a threshold of £30,000 per quality-adjusted life-year, even using the list price of £243 per injection. Conclusions Wider adoption of bevacizumab for the treatment of macular oedema due to central retinal vein occlusion could result in substantial savings to healthcare systems and deliver similar health-related quality of life. However, patients, funders and ophthalmologists should be fully aware that LEAVO could not demonstrate that bevacizumab is non-inferior to the licensed agents.

Introduction Adaptive designs allow changes to an ongoing trial based on prespecified early examinations of accrued data. Opportunities are potentially being missed to incorporate health economic considerations into the design of these studies. Methods We describe how to estimate the expected value of sample information for group sequential design adaptive trials. We operationalize this approach in a hypothetical case study using data from a pilot trial. We report the expected value of sample information and expected net benefit of sampling results for 5 design options for the future full-scale trial including the fixed-sample-size design and the group sequential design using either the Pocock stopping rule or the O’Brien-Fleming stopping rule with 2 or 5 analyses. We considered 2 scenarios relating to 1) using the cost-effectiveness model with a traditional approach to the health economic analysis and 2) adjusting the cost-effectiveness analysis to incorporate the bias-adjusted maximum likelihood estimates of trial outcomes to account for the bias that can be generated in adaptive trials. Results The case study demonstrated that the methods developed could be successfully applied in practice. The results showed that the O’Brien-Fleming stopping rule with 2 analyses was the most efficient design with the highest expected net benefit of sampling in the case study. Conclusions Cost-effectiveness considerations are unavoidable in budget-constrained, publicly funded health care systems, and adaptive designs can provide an alternative to costly fixed-sample-size designs. We recommend that when planning a clinical trial, expected value of sample information methods be used to compare possible adaptive and nonadaptive trial designs, with appropriate adjustment, to help justify the choice of design characteristics and ensure the cost-effective use of research funding. Highlights Opportunities are potentially being missed to incorporate health economic considerations into the design of adaptive clinical trials. Existing expected value of sample information analysis methods can be extended to compare possible group sequential and nonadaptive trial designs when planning a clinical trial. We recommend that adjusted analyses be presented to control for the potential impact of the adaptive designs and to maintain the accuracy of the calculations. This approach can help to justify the choice of design characteristics and ensure the cost-effective use of limited research funding.

IntroductionAdaptive designs offer a flexible approach, allowing changes to a trial based on examinations of the data as it progresses. Adaptive clinical trials are becoming a popular choice, as the prudent use of finite research budgets and accurate decision-making are priorities for healthcare providers around the world. The methods of health economics, which aim to maximise the health gained for money spent, could be incorporated into the design and analysis of adaptive clinical trials to make them more efficient. We aimed to understand the perspectives of stakeholders in health technology assessments to inform recommendations for the use of health economics in adaptive clinical trials.MethodsA qualitative study explored the attitudes of key stakeholders—including researchers, decision-makers and members of the public—towards the use of health economics in the design and analysis of adaptive clinical trials. Data were collected using interviews and focus groups (29 participants). A framework analysis was used to identify themes in the transcripts.ResultsIt was considered that answering the clinical research question should be the priority in a clinical trial, notwithstanding the importance of cost-effectiveness for decision-making. Concerns raised by participants included handling the volatile nature of cost data at interim analyses; implementing this approach in global trials; resourcing adaptive trials which are designed and adapted based on health economic outcomes; and training stakeholders in these methods so that they can be implemented and appropriately interpreted.ConclusionThe use of health economics in the design and analysis of adaptive clinical trials has the potential to increase the efficiency of health technology assessments worldwide. Recommendations are made concerning the development of methods allowing the use of health economics in adaptive clinical trials, and suggestions are given to facilitate their implementation in practice.

Background Adaptive designs offer great promise in improving the efficiency and patient-benefit of clinical trials. An important barrier to further increased use is a lack of understanding about which additional resources are required to conduct a high-quality adaptive clinical trial, compared to a traditional fixed design. The Costing Adaptive Trials (CAT) project investigated which additional resources may be required to support adaptive trials. Methods We conducted a mock costing exercise amongst seven Clinical Trials Units (CTUs) in the UK. Five scenarios were developed, derived from funded clinical trials, where a non-adaptive version and an adaptive version were described. Each scenario represented a different type of adaptive design. CTU staff were asked to provide the costs and staff time they estimated would be needed to support the trial, categorised into specified areas (e.g. statistics, data management, trial management). This was calculated separately for the non-adaptive and adaptive version of the trial, allowing paired comparisons. Interviews with 10 CTU staff who had completed the costing exercise were conducted by qualitative researchers to explore reasons for similarities and differences. Results Estimated resources associated with conducting an adaptive trial were always (moderately) higher than for the non-adaptive equivalent. The median increase was between 2 and 4% for all scenarios, except for sample size re-estimation which was 26.5% (as the adaptive design could lead to a lengthened study period). The highest increase was for statistical staff, with lower increases for data management and trial management staff. The percentage increase in resources varied across different CTUs. The interviews identified possible explanations for differences, including (1) experience in adaptive trials, (2) the complexity of the non-adaptive and adaptive design, and (3) the extent of non-trial specific core infrastructure funding the CTU had. Conclusions This work sheds light on additional resources required to adequately support a high-quality adaptive trial. The percentage increase in costs for supporting an adaptive trial was generally modest and should not be a barrier to adaptive designs being cost-effective to use in practice. Informed by the results of this research, guidance for investigators and funders will be developed on appropriately resourcing adaptive trials.

Background Licensed ranibizumab (0.5 mg/0.05 ml Lucentis®; Novartis International AG, Basel, Switzerland) and aflibercept (2 mg/0.05 ml Eylea®; Bayer AG, Leverkusen, Germany) and unlicensed bevacizumab (1.25 mg/0.05 ml Avastin®; F. Hoffmann-La Roche AG, Basel, Switzerland) are used to treat macula oedema due to central retinal vein occlusion, but their relative clinical effectiveness, cost-effectiveness and impact on the UK NHS and Personal Social Services have never been directly compared over the typical disease treatment period. Objective The objective was to compare the clinical effectiveness and cost-effectiveness of three intravitreal antivascular endothelial growth factor agents for the management of macula oedema due to central retinal vein occlusion. Design This was a three-arm, double-masked, randomised controlled non-inferiority trial. Setting The trial was set in 44 UK NHS ophthalmology departments, between 2014 and 2018. Participants A total of 463 patients with visual impairment due to macula oedema secondary to central retinal vein occlusion were included in the trial. Interventions The participants were treated with repeated intravitreal injections of ranibizumab (n = 155), aflibercept (n = 154) or bevacizumab (n = 154). Main outcome measures The primary outcome was an increase in the best corrected visual acuity letter score from baseline to 100 weeks in the trial eye. The null hypothesis that aflibercept and bevacizumab are each inferior to ranibizumab was tested with a non-inferiority margin of –5 visual acuity letters over 100 weeks. Secondary outcomes included additional visual acuity, and imaging outcomes, Visual Function Questionnaire-25, EuroQol-5 Dimensions with and without a vision bolt-on, and drug side effects. Cost-effectiveness was estimated using treatment costs and Visual Function Questionnaire-Utility Index to measure quality-adjusted life-years. Results The adjusted mean changes at 100 weeks in the best corrected visual acuity letter scores were as follows – ranibizumab, 12.5 letters (standard deviation 21.1 letters); aflibercept, 15.1 letters (standard deviation 18.7 letters); and bevacizumab, 9.8 letters (standard deviation 21.4 letters). Aflibercept was non-inferior to ranibizumab in the intention-to-treat population (adjusted mean best corrected visual acuity difference 2.23 letters, 95% confidence interval –2.17 to 6.63 letters; p = 0.0006), but not superior. The study was unable to demonstrate that bevacizumab was non-inferior to ranibizumab in the intention-to-treat population (adjusted mean best corrected visual acuity difference –1.73 letters, 95% confidence interval –6.12 to 2.67 letters; p = 0.071). A post hoc analysis was unable to demonstrate that bevacizumab was non-inferior to aflibercept in the intention-to-treat population (adjusted mean best corrected visual acuity difference was –3.96 letters, 95% confidence interval –8.34 to 0.42 letters; p = 0.32). All per-protocol population results were the same. Fewer injections were required with aflibercept (10.0) than with ranibizumab (11.8) (difference in means –1.8, 95% confidence interval –2.9 to –0.8). A post hoc analysis showed that more bevacizumab than aflibercept injections were required (difference in means 1.6, 95% confidence interval 0.5 to 2.7). There were no new safety concerns. The model- and trial-based cost-effectiveness analyses estimated that bevacizumab was the most cost-effective treatment at a threshold of £20,000–30,000 per quality-adjusted life-year. Limitations The comparison of aflibercept and bevacizumab was a post hoc analysis. Conclusion The study showed aflibercept to be non-inferior to ranibizumab. However, the possibility that bevacizumab is worse than ranibizumab and aflibercept by 5 visual acuity letters cannot be ruled out. Bevacizumab is an economically attractive treatment alternative and would lead to substantial cost savings to the NHS and other health-care systems. However, uncertainty about its relative effectiveness should be discussed comprehensively with patients, their representatives and funders before treatment is considered. Future work To obtain extensive patient feedback and discuss with all stakeholders future bevacizumab NHS use. Trial registration Current Controlled Trials ISRCTN13623634. Funding This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 38. See the NIHR Journals Library website for further project information.

Objectives Mappings to convert clinical measures to preference-based measures of health such as the EQ-5D-3L are sometimes required in cost-utility analyses. We developed mappings to convert best-corrected visual acuity (BCVA) to the EQ-5D-3L, the EQ-5D-3L with a vision bolt-on (EQ-5D V), and the Visual Functioning Questionnaire-Utility Index (VFQ-UI) in patients with macular edema caused by central retinal vein occlusion. Methods We used data from Lucentis, Eylea, Avastin in vein occlusion (LEAVO), which is a phase-3 randomized controlled trial comparing ranibizumab, aflibercept, and bevacizumab in 463 patients with observations at 6 time points. We estimated adjusted limited dependent variable mixture models consisting of 1 to 4 distributions (components) using BCVA in each eye, age, and sex to predict utility within the components and BCVA as a determinant of component membership. We compared model fit using mean error, mean absolute error, root mean square error, Akaike information criteria, Bayesian information criteria, and visual inspection of mean predicted and observed utilities and cumulative distribution functions. Results Mean utility scores were 0.82 for the EQ-5D-3L, 0.79 for the EQ-5D V, and 0.88 for the VFQ-UI. The best-fitting models for the EQ-5D and EQ-5D V had 2 components (with means of approximately 0.44 and 0.85), and the best-fitting model for VFQ-UI had 3 components (with means of approximately 0.95, 0.74, and 0.90). Conclusions Models with multiple components better predict utility than those with single components. This article provides a valuable addition to the literature, in which previous mappings in visual acuity have been limited to linear regressions, resulting in unfounded assumptions about the distribution of the dependent variable., Highlights • Adjusted, limited dependent variable mixture models with multiple components better predict utility using the EQ-5D-3L with and without vision bolt-on and Visual Functioning Questionnaire-Utility Index (VFQ-UI) than mappings with only 1 component. Previous mappings in visual acuity have considered only linear regression using ordinary least squares (1 component). • Our mappings can be used to predict 3 different measures of utility as a function of age, sex, and best-corrected visual acuity in both eyes. These variables are routinely included in economic evaluations of visual disorders.

Objective\ud An adaptive design uses data collected as a clinical trial progresses to inform modifications to the trial. Hence, adaptive designs and health economics aim to facilitate efficient and accurate decision making. Nevertheless, it is unclear whether the methods are considered together in the design, analysis, and reporting of trials. This review aims to establish how health economic outcomes are used in the design, analysis, and reporting of adaptive designs.\ud \ud Methods\ud Registered and published trials up to August 2016 with an adaptive design and health economic analysis were identified. The use of health economics in the design, analysis, and reporting was assessed. Summary statistics are presented and recommendations formed based on the research team's experiences and a practical interpretation of the results.\ud \ud Results\ud Thirty-seven trials with an adaptive design and health economic analysis were identified. It was not clear whether the health economic analysis accounted for the adaptive design in 17/37 trials where this was thought necessary, nor whether health economic outcomes were used at the interim analysis for 18/19 of trials with results. The reporting of health economic results was suboptimal for the (17/19) trials with published results.\ud \ud Conclusions\ud Appropriate consideration is rarely given to the health economic analysis of adaptive designs. Opportunities to use health economic outcomes in the design and analysis of adaptive trials are being missed. Further work is needed to establish whether adaptive designs and health economic analyses can be used together to increase the efficiency of health technology assessments without compromising accuracy.

Background In individually randomised trials we might expect interventions delivered in groups or by care providers to result in clustering of outcomes for participants treated in the same group or by the same care provider. In partially nested randomised controlled trials (pnRCTs) this clustering only occurs in one trial arm, commonly the intervention arm. It is important to measure and account for between-cluster variability in trial design and analysis. We compare analysis approaches for pnRCTs with continuous outcomes, investigating the impact on statistical inference of cluster sizes, coding of the non-clustered arm, intracluster correlation coefficient (ICCs), and differential variance between intervention and control arm, and provide recommendations for analysis. Methods We performed a simulation study assessing the performance of six analysis approaches for a two-arm pnRCT with a continuous outcome. These include: linear regression model; fully clustered mixed-effects model with singleton clusters in control arm; fully clustered mixed-effects model with one large cluster in control arm; fully clustered mixed-effects model with pseudo clusters in control arm; partially nested homoscedastic mixed effects model, and partially nested heteroscedastic mixed effects model. We varied the cluster size, number of clusters, ICC, and individual variance between the two trial arms. Results All models provided unbiased intervention effect estimates. In the partially nested mixed-effects models, methods for classifying the non-clustered control arm had negligible impact. Failure to account for even small ICCs resulted in inflated Type I error rates and over-coverage of confidence intervals. Fully clustered mixed effects models provided poor control of the Type I error rates and biased ICC estimates. The heteroscedastic partially nested mixed-effects model maintained relatively good control of Type I error rates, unbiased ICC estimation, and did not noticeably reduce power even with homoscedastic individual variances across arms. Conclusions In general, we recommend the use of a heteroscedastic partially nested mixed-effects model, which models the clustering in only one arm, for continuous outcomes similar to those generated under the scenarios of our simulations study. However, with few clusters (3–6), small cluster sizes (5–10), and small ICC (≤0.05) this model underestimates Type I error rates and there is no optimal model. Electronic supplementary material The online version of this article (10.1186/s12874-018-0559-x) contains supplementary material, which is available to authorized users.

Adaptive designs can make clinical trials more flexible by utilising results accumulating in the trial to modify the trial’s course in accordance with pre-specified rules. Trials with an adaptive design are often more efficient, informative and ethical than trials with a traditional fixed design since they often make better use of resources such as time and money, and might require fewer participants. Adaptive designs can be applied across all phases of clinical research, from early-phase dose escalation to confirmatory trials. The pace of the uptake of adaptive designs in clinical research, however, has remained well behind that of the statistical literature introducing new methods and highlighting their potential advantages. We speculate that one factor contributing to this is that the full range of adaptations available to trial designs, as well as their goals, advantages and limitations, remains unfamiliar to many parts of the clinical community. Additionally, the term adaptive design has been misleadingly used as an all-encompassing label to refer to certain methods that could be deemed controversial or that have been inadequately implemented. We believe that even if the planning and analysis of a trial is undertaken by an expert statistician, it is essential that the investigators understand the implications of using an adaptive design, for example, what the practical challenges are, what can (and cannot) be inferred from the results of such a trial, and how to report and communicate the results. This tutorial paper provides guidance on key aspects of adaptive designs that are relevant to clinical triallists. We explain the basic rationale behind adaptive designs, clarify ambiguous terminology and summarise the utility and pitfalls of adaptive designs. We discuss practical aspects around funding, ethical approval, treatment supply and communication with stakeholders and trial participants. Our focus, however, is on the interpretation and reporting of results from adaptive design trials, which we consider vital for anyone involved in medical research. We emphasise the general principles of transparency and reproducibility and suggest how best to put them into practice.

A sample size justification is a vital part of any trial design. However, estimating the number of participants required to give a meaningful result is not always straightforward. A number of components are required to facilitate a suitable sample size calculation. In this paper, the steps for conducting sample size calculations for non-inferiority and equivalence trials are summarised. Practical advice and examples are provided that illustrate how to carry out the calculations by hand and using the app SampSize.

Background \ud Substantial amounts of public funds are invested in health research worldwide. Publicly funded randomised controlled trials (RCTs) often recruit participants at a slower than anticipated rate. Many trials fail to reach their planned sample size within the envisaged trial timescale and trial funding envelope.\ud \ud Objectives \ud To review the consent, recruitment and retention rates for single and multicentre randomised control trials funded and published by the UK's National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme.\ud \ud Data sources and study selection \ud HTA reports of individually randomised single or multicentre RCTs published from the start of 2004 to the end of April 2016 were reviewed.\ud \ud Data extraction \ud Information was extracted, relating to the trial characteristics, sample size, recruitment and retention by two independent reviewers.\ud \ud Main outcome measures \ud Target sample size and whether it was achieved; recruitment rates (number of participants recruited per centre per month) and retention rates (randomised participants retained and assessed with valid primary outcome data).\ud \ud Results \ud This review identified 151 individually RCTs from 787 NIHR HTA reports. The final recruitment target sample size was achieved in 56% (85/151) of the RCTs and more than 80% of the final target sample size was achieved for 79% of the RCTs (119/151). The median recruitment rate (participants per centre per month) was found to be 0.92 (IQR 0.43–2.79) and the median retention rate (proportion of participants with valid primary outcome data at follow-up) was estimated at 89% (IQR 79–97%).\ud \ud Conclusions \ud There is considerable variation in the consent, recruitment and retention rates in publicly funded RCTs. Investigators should bear this in mind at the planning stage of their study and not be overly optimistic about their recruitment projections.

Background: Adaptive design clinical trials use preplanned interim analyses to determine whether studies should be stopped or modified before recruitment is complete. Emergency medicine trials are well suited to these designs as many have a short time to primary outcome relative to the length of recruitment. We hypothesised that the majority of published emergency medicine trials have the potential to use a simple adaptive trial design.\ud \ud Methods: We reviewed clinical trials published in three emergency medicine journals between January 2003 and December 2013. We determined the proportion that used an adaptive design as well as the proportion that could have used a simple adaptive design based on the time to primary outcome and length of recruitment.\ud \ud Results: Only 19 of 188 trials included in the review were considered to have used an adaptive trial design. A total of 154/165 trials that were fixed in design had the potential to use an adaptive design.\ud \ud Conclusions: Currently, there seems to be limited uptake in the use of adaptive trial designs in emergency medicine despite their potential benefits to save time and resources. Failing to take advantage of adaptive designs could be costly to patients and research. It is recommended that where practical and logistical considerations allow, adaptive designs should be used for all emergency medicine clinical trials.

Adaptive designs have the potential to improve efficiency in the evaluation of new medical treatments in comparison to traditional fixed sample size designs. However, they are still not widely used in practice in clinical research. Little research has been conducted to investigate what adaptive designs are being undertaken. This review highlights the current state of registered adaptive designs and their characteristics. The review looked at phase II, II/III and III trials registered on ClinicalTrials.gov from 29 February 2000 to 1 June 2014, supplemented with trials from the National Institute for Health Research register and known adaptive trials. A range of adaptive design search terms were applied to the trials extracted from each database. Characteristics of the adaptive designs were then recorded including funder, therapeutic area and type of adaptation. The results in the paper suggest that the use of adaptive designs has increased. They seem to be most often used in phase II trials and in oncology. In phase III trials, the most popular form of adaptation is the group sequential design. The review failed to capture all trials with adaptive designs, which suggests that the reporting of adaptive designs, such as in clinical trials registers, needs much improving. We recommend that clinical trial registers should contain sections dedicated to the type and scope of the adaptation and that the term ‘adaptive design’ should be included in the trial title or at least in the brief summary or design sections. Electronic supplementary material The online version of this article (doi:10.1186/s13063-016-1273-9) contains supplementary material, which is available to authorized users.

In an individually randomised controlled trial (iRCT) where the intervention is delivered by a health professional/therapist it seems likely that the effectiveness of the intervention could depend on the skill of the therapist delivering it. This leads to a potential clustering of the outcomes for the patients treated by the same therapist. If outcomes are clustered then the usual statistical methods for analysing RCT data may not be appropriate as they assume that outcomes observed on different participants are independent. In some iRCTs there is clustering in only one arm; where the therapist only treats participants in one arm of the trial but there is no equivalent clustering in the control arm. Several strategies have been proposed for this problem: ignoring the clustering; imposing clusters in the control arm and fitting a fixed or random effects model, or using a partially clustered approach where only the clustering in the treatment arm is modelled. This talk will describe and compare the statistical methods for analysing continuous outcomes from an iRCT with some element of clustering in one arm. Four confirmatory case studies (with a combined 1,167 randomised participants) will be analysed using each of the methods: specialist clinics for the treatment of venous leg ulcers (Morrell et al. 1998), acupuncture for low back pain (Thomas et al. 2006), cost-effectiveness of community postnatal support workers (CPSW) (Morrell et al. 2000), and Putting Life in Years (PLINY) (Mountain et al. 2014). Recommendations for the best and most practical approaches will be made.

A sample size justification is a vital step when designing any trial. However, estimating the number of participants required to give a meaningful result is not always straightforward. A number of components are required to facilitate a suitable sample size calculation. In this paper, the general steps are summarised for conducting sample size calculations with practical advice and guidance on how to utilise the app SampSize.

A sample size justification is a vital part of any investigation. However, estimating the number of participants required to give meaningful results is not always straightforward. A number of components are required to facilitate a suitable sample size calculation. In this paper, the steps for conducting sample size calculations for superiority trials are summarised. Practical advice and examples are provided illustrating how to carry out the calculations by hand and using the app SampSize.