Search

The National Research Council (NRC) Expert Panel Report on Prevention and Treatment of Missing Data in Clinical Trials highlighted the need for clearly defining objectives and estimands. That report sparked considerable discussion and literature on estimands and how to choose them. Importantly, consideration moved beyond missing data to include all postrandomization events that have implications for estimating quantities of interest (intercurrent events, aka ICEs). The ICH E9(R1) draft addendum builds on that research to outline key principles in choosing estimands for clinical trials, primarily with focus on confirmatory trials. This paper provides additional insights, perspectives, details, and examples to help put ICH E9(R1) into practice. Specific areas of focus include how the perspectives of different stakeholders influence the choice of estimands; the role of randomization and the intention-to-treat principle; defining the causal effects of a clearly defined treatment regimen, along with the implications this has for trial design and the generalizability of conclusions; detailed discussion of strategies for handling ICEs along with their implications and assumptions; estimands for safety objectives, time-to-event endpoints, early-phase and one-arm trials, and quality of life endpoints; and realistic examples of the thought process involved in defining estimands in specific clinical contexts.

The draft ICH E9(R1) addendum stipulates that an estimator should align with its associated estimand and yield an estimate that facilitates reliable interpretations. The addendum further stipulates that assumptions should be justifiable and plausible, and that the extent of assumptions is an important consideration for whether an estimate will be robust because assumptions are often unverifiable. The draft addendum specifies 5 strategies for dealing with intercurrent events. The intent of this paper is to provide conceptual considerations and technical details for various estimators that align with these strategies. We include focus on how the nature and extent of assumptions influences the potential robustness of the various estimators. The content reflects the knowledge, experience, and opinions of the Drug Information Association's Scientific Working Group on Missing Data. This group includes experienced statisticians from across industry and academia, primarily in the US and European Union.

Since the introduction of the propensity score (PS), methods for estimating treatment effects with observational data have received growing attention in the literature. Recent research has added substantially to the number of available statistical approaches for controlling confounding in such analyses. However, researchers need guidance to decide on the optimal analytic strategy for any given scenario. To address this gap, we conducted simulations evaluating both well-established methods (regression, PS weighting, stratification, and matching) and more recently proposed approaches (tree-based methods, local control, entropy balancing, genetic matching, prognostic scoring). The simulation scenarios included tree-based and smooth regression models as true data-generation mechanisms. We evaluated an extensive number of analysis strategies combining different treatment choices and outcome models. Key findings include 1) the lack of a single best strategy across all potential scenarios; 2) the importance of appropriately addressing interactions in the treatment choice model and/or outcome model; and 3) a tree-structured treatment choice model and a polynomial outcome model with second-order interactions performed well. One limitation to this initial assessment is the lack of heterogeneous simulation scenarios allowing treatment effects to vary by patient.

Over the past years, significant progress has been made in developing statistically rigorous methods to implement clinically interpretable sensitivity analyses for assumptions about the missingness mechanism in clinical trials for continuous and (to a lesser extent) for binary or categorical endpoints. Studies with time-to-event outcomes have received much less attention. However, such studies can be similarly challenged with respect to the robustness and integrity of primary analysis conclusions when a substantial number of subjects withdraw from treatment prematurely prior to experiencing an event of interest. We discuss how the methods that are widely used for primary analyses of time-to-event outcomes could be extended in a clinically meaningful and interpretable way to stress-test the assumption of ignorable censoring. We focus on a 'tipping point' approach, the objective of which is to postulate sensitivity parameters with a clear clinical interpretation and to identify a setting of these parameters unfavorable enough towards the experimental treatment to nullify a conclusion that was favorable to that treatment. Robustness of primary analysis results can then be assessed based on clinical plausibility of the scenario represented by the tipping point. We study several approaches for conducting such analyses based on multiple imputation using parametric, semi-parametric, and non-parametric imputation models and evaluate their operating characteristics via simulation. We argue that these methods are valuable tools for sensitivity analyses of time-to-event data and conclude that the method based on piecewise exponential imputation model of survival has some advantages over other methods studied here. Copyright © 2016 John Wiley & Sons, Ltd., (Copyright © 2016 John Wiley & Sons, Ltd.)

It is important to understand the effects of a drug as actually taken (effectiveness) and when taken as directed (efficacy). The primary objective of this investigation was to assess the statistical performance of a method referred to as placebo multiple imputation (pMI) as an estimator of effectiveness and as a worst reasonable case sensitivity analysis in assessing efficacy. The pMI method assumes the statistical behavior of placebo- and drug-treated patients after dropout is the statistical behavior of placebo-treated patients. Thus, in the effectiveness context, pMI assumes no pharmacological benefit of the drug after dropout. In the efficacy context, pMI is a specific form of a missing not at random analysis expected to yield a conservative estimate of efficacy. In a simulation study with 18 scenarios, the pMI approach generally provided unbiased estimates of effectiveness and conservative estimates of efficacy. However, the confidence interval coverage was consistently greater than the nominal coverage rate. In contrast, last and baseline observation carried forward (LOCF and BOCF) were conservative in some scenarios and anti-conservative in others with respect to efficacy and effectiveness. As expected, direct likelihood (DL) and standard multiple imputation (MI) yielded unbiased estimates of efficacy and tended to overestimate effectiveness in those scenarios where a drug effect existed. However, in scenarios with no drug effect, and therefore where the true values for both efficacy and effectiveness were zero, DL and MI yielded unbiased estimates of efficacy and effectiveness.

Objectives: To better evaluate the effectiveness of antidepressant drugs in the treatment of major depression in clinical practice., Methods: A simulation experiment was used to illustrate an application of marginal structural models (MSMs) via inverse probability of treatment weighting (IPTW) approach in the context of non-randomized data on N = 1,000 depressed subjects, initially subjected to "watchful waiting". In simulation we assumed that subjects with worse intermediate outcome have a higher probability of being subsequently assigned to antidepressant treatment while those who receive antidepressant treatment are more likely to reach remission and less likely to reach relapse state. The outcomes from multiple (500) simulated data sets are analyzed using simple unadjusted analysis based on logistic regression and using MSM., Results: In contrast to unadjusted analysis, but consistent with the treatment assumptions, using MSM via IPTW results in strong evidence of the effectiveness of the antidepressant treatment. Furthermore MSM via IPTW substantially reduces the probability of wrongly rejecting the null hypothesis. However, the instability of weights due to the sparse data and incorrectly specified MSM may still result in inflation of Type I error rates., Conclusions: MSMs may allow evaluating the causal effects associated with antidepressant treatment from the data observed in clinical practice.

Assessing dose response from flexible-dose clinical trials is problematic. The true dose effect may be obscured and even reversed in observed data because dose is related to both previous and subsequent outcomes. To remove selection bias, we propose marginal structural models, inverse probability of treatment-weighting (IPTW) methodology. Potential clinical outcomes are compared across dose groups using a marginal structural model (MSM) based on a weighted pooled repeated measures analysis (generalized estimating equations with robust estimates of standard errors), with dose effect represented by current dose and recent dose history, and weights estimated from the data (via logistic regression) and determined as products of (i) inverse probability of receiving dose assignments that were actually received and (ii) inverse probability of remaining on treatment by this time. In simulations, this method led to almost unbiased estimates of true dose effect under various scenarios. Results were compared with those obtained by unweighted analyses and by weighted analyses under various model specifications. The simulation showed that the IPTW MSM methodology is highly sensitive to model misspecification even when weights are known. Practitioners applying MSM should be cautious about the challenges of implementing MSM with real clinical data. Clinical trial data are used to illustrate the methodology., (Copyright © 2012 John Wiley & Sons, Ltd.)

An important evolution in the missing data arena has been the recognition of need for clarity in objectives. The objectives of primary focus in clinical trials can often be categorized as assessing efficacy or effectiveness. The present investigation illustrated a structured framework for choosing estimands and estimators when testing investigational drugs to treat the symptoms of chronic illnesses. Key issues were discussed and illustrated using a reanalysis of the confirmatory trials from a new drug application in depression. The primary analysis used a likelihood-based approach to assess efficacy: mean change to the planned endpoint of the trial assuming patients stayed on drug. Secondarily, effectiveness was assessed using a multiple imputation approach. The imputation model-derived solely from the placebo group-was used to impute missing values for both the drug and placebo groups. Therefore, this so-called placebo multiple imputation (a.k.a. controlled imputation) approach assumed patients had reduced benefit from the drug after discontinuing it. Results from the example data provided clear evidence of efficacy for the experimental drug and characterized its effectiveness. Data after discontinuation of study medication were not required for these analyses. Given the idiosyncratic nature of drug development, no estimand or approach is universally appropriate. However, the general practice of pairing efficacy and effectiveness estimands may often be useful in understanding the overall risks and benefits of a drug. Controlled imputation approaches, such as placebo multiple imputation, can be a flexible and transparent framework for formulating primary analyses of effectiveness estimands and sensitivity analyses for efficacy estimands., (Copyright © 2012 John Wiley & Sons, Ltd.)