Search

A surrogate endpoint in a randomized clinical trial is an endpoint that occurs after randomization and before the true, clinically meaningful, endpoint that yields conclusions about the effect of treatment on true endpoint. A surrogate endpoint can accelerate the evaluation of new treatments but at the risk of misleading conclusions. Therefore, criteria are needed for deciding whether to use a surrogate endpoint in a new trial. For the meta-analytic setting of multiple previous trials, each with the same pair of surrogate and true endpoints, this article formulates five criteria for using a surrogate endpoint in a new trial to predict the effect of treatment on the true endpoint in the new trial The first two criteria, which are easily computed from a zero-intercept linear random effects model, involve statistical considerations: an acceptable sample size multiplier and an acceptable prediction separation score. The remaining three criteria involve clinical and biological considerations: similarity of biological mechanisms of treatments between the new trial and previous trials, similarity of secondary treatments following the surrogate endpoint between the new trial and previous trials, and a negligible risk of harmful side effects arising after the observation of the surrogate endpoint in the new trial. These five criteria constitute an appropriately high bar for using a surrogate endpoint to make a definitive treatment recommendation.

An important question in the evaluation of an additional risk prediction marker is how to interpret a small increase in the area under the receiver operating characteristic curve (AUC). Many researchers believe that a change in AUC is a poor metric because it increases only slightly with the addition of a marker with a large odds ratio. Because it is not possible on purely statistical grounds to choose between the odds ratio and AUC, we invoke decision analysis, which incorporates costs and benefits. For example, a timely estimate of the risk of later non-elective operative delivery can help a woman in labor decide if she wants an early elective cesarean section to avoid greater complications from possible later non-elective operative delivery. A basic risk prediction model for later non-elective operative delivery involves only antepartum markers. Because adding intrapartum markers to this risk prediction model increases AUC by 0.02, we questioned whether this small improvement is worthwhile. A key decision-analytic quantity is the risk threshold, here the risk of later non-elective operative delivery at which a patient would be indifferent between an early elective cesarean section and usual care. For a range of risk thresholds, we found that an increase in the net benefit of risk prediction requires collecting intrapartum marker data on 68 to 124 women for every correct prediction of later non-elective operative delivery. Because data collection is non-invasive, this test tradeoff of 68 to 124 is clinically acceptable, indicating the value of adding intrapartum markers to the risk prediction model.

I was delighted that Judea Pearl wrote a commentary [1] on my paper “Causal inference, probability theory, and graphical insights”[2] because it provides a springboard for improving clarity. Judea Pearl and I tackle the problem of causal inference from different research backgrounds which may explain the difference in our viewpoints.

In some two-arm randomized trials, some participants receive the treatment assigned to the other arm as a result of technical problems, refusal of a treatment invitation, or a choice of treatment in an encouragement design. In some before-and-after studies, the availability of a new treatment changes from one time period to this next. Under assumptions that are often reasonable, the latent class instrumental variable (IV) method estimates the effect of treatment received in the aforementioned scenarios involving all-or-none compliance and all-or-none availability. Key aspects are four initial latent classes (sometimes called principal strata) based on treatment received if in each randomization group or time period, the exclusion restriction assumption (in which randomization group or time period is an instrumental variable), the monotonicity assumption (which drops an implausible latent class from the analysis), and the estimated effect of receiving treatment in one latent class (sometimes called efficacy, the local average treatment effect, or the complier average causal effect). Since its independent formulations in the biostatistics and econometrics literatures, the latent class IV method (which has no well-established name) has gained increasing popularity. We review the latent class IV method from a clinical and biostatistical perspective, focusing on underlying assumptions, methodological extensions, and applications in our fields of obstetrics and cancer research.

Many long-term studies of the early detection of cancer involve serial observations of precancerous lesions and information as to whether or not the subject was diagnosed with cancer during the study period. Often the purpose of these studies is to decide whether or not the precancerous lesion should be studied in a future trial as a trigger for early intervention. A general approach to the analysis of cancer biomarkers is to estimate false and true positive rates to determine if they fall in the target region of those false and true positives that indicate promise for further study. The challenge with analysing serial data on precancerous lesions is estimating false and true positive rates when the number of observations varies among subjects. To solve this problem, we propose a Markov chain model in reverse time. The methodology is illustrated using serial observations of precancerous lesions found on sputum cytology.

We propose a new, less costly, design to test the equivalence of digital versus analogue mammography in terms of sensitivity and specificity. Because breast cancer is a rare event among asymptomatic women, the sample size for testing equivalence of sensitivity is larger than that for testing equivalence of specificity. Hence calculations of sample size are based on sensitivity. With the proposed design it is possible to achieve the same power as a completely paired design by increasing the number of less costly analogue mammograms and not giving the more expensive digital mammograms to some randomly selected subjects who are negative on the analogue mammogram. The key idea is that subjects who are negative on the analogue mammogram are unlikely to have cancer and hence contribute less information for estimating sensitivity than subjects who are positive on the analogue mammogram. To ascertain disease state among subjects not biopsied, we propose another analogue mammogram at a later time determined by a natural history model. The design differs from a double sampling design because it compares two imperfect tests instead of combining information from a perfect and imperfect test.

Causal inference from observational studies is a fundamental topic in biostatistics. The causal graph literature typically views probability theory as insufficient to express causal concepts in observational studies. In contrast, the view here is that probability theory is a desirable and sufficient basis for many topics in causal inference for the following two reasons. First, probability theory is generally more flexible than causal graphs: Besides explaining such causal graph topics as M-bias (adjusting for a collider) and bias amplification and attenuation (when adjusting for instrumental variable), probability theory is also the foundation of the paired availability design for historical controls, which does not fit into a causal graph framework. Second, probability theory is the basis for insightful graphical displays including the BK-Plot for understanding Simpson's paradox with a binary confounder, the BK2-Plot for understanding bias amplification and attenuation in the presence of an unobserved binary confounder, and the PAD-Plot for understanding the principal stratification component of the paired availability design.

Eng [1] discussed four designs for randomized trials to evaluate the use of a single biomarker to target treatments. Design 1(marker interaction) tests for the marker and then separately randomizes participants with each level of the marker to treatment A or treatment B. Design 2 (marker-based strategy) randomizes participants to MB (marker-based) strategy or treatment B, where MB strategy is treatment A if the marker is positive and treatment B otherwise. Design 3 (modified marker-based strategy) randomizes participants to MB strategy or further randomization to treatment A or treatment B. Design 4 (reverse marker-based strategy) randomizes participants to MB strategy or RM (reverse marker) strategy, where RM strategy is treatment B if the marker is positive and treatment A otherwise. Eng noted that “it is of primary importance that the study designer is clear on what quantity best represents the research question: interaction, the marker strategy, or the subgroup treatment effect.” With the goal of “testing the interaction between a marker and its intended treatment,” Eng demonstrated the advantages of Design 4. However, for addressing research questions involving marker strategy and subgroup treatment effect, designs other than Design 4 are more appropriate. Let Y denote binary response, T=A, B denote treatment, and M =M+, M− denote the outcomes of a binary marker The four basic quantities in Designs 1, 2, 3, and 4 are pr(Y | MB strategy), pr(Y | RM strategy), pr(Y | treatment A), and pr(Y | treatment B). Also let π = pr(M=M+), θi = pr(Y | T=i), θij = pr(Y | T=i, M=j), and pr(Y | MB strategy) ≡ Φ1 = π θA(M+) + (1 − π) θB(M−), pr(Y | RM strategy) ≡ Φ4 = π θB(M+) + (1 − π) θA(M−). Noting that θi = π θi(M+) + (1− π) θi(M−) and extending previous analyses [2, 3], research questions related to marker strategy versus treatment involve estimating pr(Y|MBstrategy)−pr(Y|treatmentB)=Φ1−θB=π(θA(M+)−θB(M+)), (1) pr(Y|MBstrategy)−pr(Y|treatmentA)=Φ1−θA=(1−π)(θB(M−)−θA(M−)), (2) pr(Y|RMstrategy)−pr(Y|treatmentB)=Φ4−θB=(1−π)(θA(M−)−θB(M−)), (3) pr(Y|RMstrategy)−pr(Y|treatmentA)=Φ4−θA=π(θB(M+)−θA(M+)). (4) Research questions related to subgroup treatment effect involve estimating pr(Y|treatmentA,M+)−pr(Y|treatmentB,M+)=θA(M+)−θB(M+), (5) pr(Y|treatmentA,M−)−pr(Y|treatmentB,M−)=θA(M−)−θB(M−). (6) Because equations (1) –(4) are proportional to either equation (5) or (6), research questions involving subgroup treatment effect address research questions involving marker strategy. Thus for addressing research questions involving subgroup treatment effect or marker strategy, investigators need only implement designs which estimate subgroup treatment effects. Also it is desirable if these designs also estimate overall treatment effect. Three designs satisfying these desiderata are Design 1 and two designs called here Design 5 and Design 6. Design 5 randomizes participants to treatments A or B and then ascertains the biomarker in all participants [4]. Design 6 (biomarker-nested-case-control) randomizes participants to treatments A or B and then ascertains the biomarker in a random sample of stored specimens stratified by response and treatment, followed by an analysis adjusted for this sampling [3, 5] – an appealing approach when biomarker ascertainment in stored specimens is expensive or there is concern about the irreplaceable usage of stored specimens.

One method for analyzing contingency tables with missing observations is to model the missing-data mechanism using log-linear models. Previous methods for obtaining estimates (of missing counts and parameters) have required an iterative algorithm. In many cases, however, one can obtain estimates by use of a simple algebraic formula. We illustrate the method with data on smoking and birth weight.

An important question in the evaluation of an additional risk prediction marker is how to interpret a small increase in the area under the receiver operating characteristic curve (AUC). Many researchers believe that a change in AUC is a poor metric because it increases only slightly with the addition of a marker with a large odds ratio. Because it is not possible on purely statistical grounds to choose between the odds ratio and AUC, we invoke decision analysis, which incorporates costs and benefits. For example, a timely estimate of the risk of later non-elective operative delivery can help a woman in labor decide if she wants an early elective cesarean section to avoid greater complications from possible later non-elective operative delivery. A basic risk prediction model for later non-elective operative delivery involves only antepartum markers. Because adding intrapartum markers to this risk prediction model increases AUC by 0.02, we questioned whether this small improvement is worthwhile. A key decision-analytic quantity is the risk threshold, here the risk of later non-elective operative delivery at which a patient would be indifferent between an early elective cesarean section and usual care. For a range of risk thresholds, we found that an increase in the net benefit of risk prediction requires collecting intrapartum marker data on 68 to 124 women for every correct prediction of later non-elective operative delivery. Because data collection is non-invasive, this test tradeoff of 68 to 124 is clinically acceptable, indicating the value of adding intrapartum markers to the risk prediction model. Copyright (C) 2014 John Wiley & Sons, Ltd.

Collins and Huynh [1] presented an extensive review of latent class methods for diagnostic tests published since 1991. A useful topic not mentioned is the application of latent class models to the new, reference, gold standard (NRGS) problem [2]. In the NRGS problem, the new test, reference test, and gold standard are all binary, and the data consist of two 2 x 2 tables of counts: (i) new test versus reference test, and (ii) reference test versus gold standard. The goal is to estimate sensitivity and specificity of the new test relative to the gold standard. Extending the method of Gart and Buck [3] to the NRGS problem, Baker [2] formulated a latent class model and derived closed form estimates and variances for the estimated sensitivity and specificity of the new test relative to the gold standard. Baker [2] also discussed extensions to covariates.

Vickers et al make a valuable contribution to the comparison of two binary diagnostic tests by taking a decision-analytic approach [1]. They introduce relative diagnostic value which yields a threshold probability for selecting the preferred test. Two additional points are worth noting. First, although the context differs, the equation relating relative diagnostic value to the odds of prevalence and threshold odds of disease is mathematically identical to an equation for finding the optimal slope of a concave Receiver Operating Characteristic (ROC) curve when the two tests correspond to adjacent points on the ROC curve [2, 3]. The relative diagnostic value is derived by setting equal the net benefits of the two tests. The optimal slope of the concave ROC curve is found by setting equal the net benefits of tests corresponding to adjacent points on the ROC curve, which corresponds to maximizing a function by finding where its slope is zero. Second the asymptotic confidence interval for the threshold probability corresponding to the relative diagnostic value can be computed with a simple formula derived from the Multinomial-Poisson (MP) transformation [4]. The relative diagnostic value is R= n0 (c1 - b1) / {n1 (c0 - b0)}. For persons without disease, no is the total number in the sample, c0 is the number positive on the first test and negative on the second test, and b0 is the number positive on the second test and negative on the first test. For persons with disease, n1, c1, and b1 are defined analogously. Using the MP transformation applied to the logarithm of R (which may have a more symmetric distribution than R) yields, after algebraic simplification, a standard error of log(R) of selogR={−1∕n0−1∕n1+(c0+b0)∕(c0−b0)2+(c1+b1)∕(c1−b1)2}12. Let RL = exp{log(R) – 1.96 selogR} and RU = exp{log(R) + 1.96 selogR}. Let π denote the prevalence of disease. The formula for the threshold probability is p = R π / (1 + R π - π), which is denoted here by f(R, π), so the asymptotic 95% CI for p is {f(RL, π), f(RU, π)}.

The paired availability design (PAD) can reduce selection bias when it is not possible to randomize subjects. PAD consists of independent pairs of experimental and control groups. Within each pair, the intervention is the availability of treatment not its receipt. In the experimental group, the new treatment is made available to all subjects although some may not receive it. In the control group, the experimental treatment is generally not available to subjects although some may receive it in special circumstances. We present a statistic to test a null hypothesis that the receipt of intervention will increase response by a specified non-zero amount delta. We propose this design for use in a study of the effect of epidural analgesia on the rate of Caesarean section.

A composite linear model (CLM) is a matrix model for incomplete multinomial data. A CLM provides a unified approach for maximum likelihood inference which is applicable to a wide variety of problems involving incomplete multinomial data. By formulating a model as a CLM, one can simplify computation of maximum likelihood estimates and asymptotic standard errors. As an example, we use CLM to test marginal homogeneity for ordered categories, subject to both ignorable and non-ignorable missing-data mechanisms.

We discuss the design of observer agreement studies with binary assessments, with particular emphasis on the need for adequate sample size and the use of replicate observations. First, we present a method and tables for determining the sample size required for ensuring a desired precision for the estimate of the probability of disagreement between two observers. Second, for studies including replicate observations, we present a statistical model that allows estimation of the magnitude of within- and between-observer variation. We then derive sample sizes guaranteeing a specified precision for these estimates, present tables of these sample sizes and give examples of their use.

The use of time dependent covariates has allowed for incorporation into analysis of survival data intervening events that are binary and non-reversible (for example, heart transplant, initial response to chemotherapy). We can represent this type of intervening event as a three-state stochastic process with a starting state (S), an intervening state (I), and an absorbing state (D), which usually represents death. In this paper we present three procedures for calculating survivorship functions which attempt to display the prognostic significance of the time dependent covariate. The first method compares survival from baseline for the two possible paths through the stochastic process; the second method compares overall survival to survival with state I removed from the process; and, the third method compares survival for those already in state I at a landmark time x to those in state S at time x who will never enter state I. We develop discrete hazard estimates for the survival curves associated with the three methods. Two examples illustrate how these methods can yield different results and in which situations one might employ each of the three methods. Extensions to applications with reversible binary time dependent covariates and models with both baseline and time dependent covariates are suggested.