Search

Post-approval safety surveillance of medical products using observational healthcare data can help identify safety issues beyond those found in pre-approval trials. When testing sequentially as data accrue, maximum sequential probability ratio testing (MaxSPRT) is a common approach to maintaining nominal type 1 error. However, the true type 1 error may still deviate from the specified one because of systematic error due to the observational nature of the analysis. This systematic error may persist even after controlling for known confounders. Here we propose to address this issue by combing MaxSPRT with empirical calibration. In empirical calibration, we assume uncertainty about the systematic error in our analysis, the source of uncertainty commonly overlooked in practice. We infer a probability distribution of systematic error by relying on a large set of negative controls: exposure-outcome pairs where no causal effect is believed to exist. Integrating this distribution into our test statistics has previously been shown to restore type 1 error to nominal. Here we show how we can calibrate the critical value central to MaxSPRT. We evaluate this novel approach using simulations and real electronic health records, using H1N1 vaccinations during the 2009-2010 season as an example. Results show that combining empirical calibration with MaxSPRT restores nominal type 1 error. In our real-world example, adjusting for systematic error using empirical calibration has a larger impact than, and hence is just as essential as, adjusting for sequential testing using MaxSPRT. We recommend performing both, using the method described here.

An attractive feature of using a Bayesian analysis for a clinical trial is that knowledge and uncertainty about the treatment effect is summarized in a posterior probability distribution. Researchers often find probability statements about treatment effects highly intuitive and the fact that this is not accommodated in frequentist inference is a disadvantage. At the same time, the requirement to specify a prior distribution in order to obtain a posterior distribution is sometimes an artificial process that may introduce subjectivity or complexity into the analysis. This paper considers a compromise involving confidence distributions, which are probability distributions that summarize uncertainty about the treatment effect without the need for a prior distribution and in a way that is fully compatible with frequentist inference. The concept of a confidence distribution provides a posterior–like probability distribution that is distinct from, but exists in tandem with, the relative frequency interpretation of probability used in frequentist inference. Although they have been discussed for decades, confidence distributions are not well known among clinical trial statisticians and the goal of this paper is to discuss their use in analyzing treatment effects from randomized trials. As well as providing an introduction to confidence distributions, some illustrative examples relevant to clinical trials are presented, along with various case studies based on real clinical trials. It is recommended that trial statisticians consider presenting confidence distributions for treatment effects when reporting analyses of clinical trials. [ABSTRACT FROM AUTHOR]

Cost-effectiveness analysis is an essential part of the evaluation of new medical interventions. While in many studies both costs and effectiveness (eg, survival time) are censored, standard survival analysis techniques are often invalid due to the induced dependent censoring problem. We propose methods for censored cost-effectiveness data using the net-benefit regression framework, which allow covariate-adjustment and subgroup identification when comparing two intervention groups. The methods provide a straightforward way to construct cost-effectiveness acceptability curves with censored data. We also propose a more efficient doubly robust estimator of average causal incremental net benefit, which increases the likelihood that the results will represent a valid inference in observational studies. Lastly, we conduct extensive numerical studies to examine the finite-sample performance of the proposed methods, and illustrate the proposed methods with a real data example using both survival time and quality-adjusted survival time as the measures of effectiveness.

Moderate calibration, the expected event probability among observations with predicted probability z being equal to z, is a desired property of risk prediction models. Current graphical and numerical techniques for evaluating moderate calibration of risk prediction models are mostly based on smoothing or grouping the data. As well, there is no widely accepted inferential method for the null hypothesis that a model is moderately calibrated. In this work, we discuss recently-developed, and propose novel, methods for the assessment of moderate calibration for binary responses. The methods are based on the limiting distributions of functions of standardized partial sums of prediction errors converging to the corresponding laws of Brownian motion. The novel method relies on well-known properties of the Brownian bridge which enables joint inference on mean and moderate calibration, leading to a unified "bridge" test for detecting miscalibration. Simulation studies indicate that the bridge test is more powerful, often substantially, than the alternative test. As a case study we consider a prediction model for short-term mortality after a heart attack, where we provide suggestions on graphical presentation and the interpretation of results. Moderate calibration can be assessed without requiring arbitrary grouping of data or using methods that require tuning of parameters., (© 2024 The Author(s). Statistics in Medicine published by John Wiley & Sons Ltd.)

Analysis of integrated data often requires record linkage in order to join together the data residing in separate sources. In case linkage errors cannot be avoided, due to the lack a unique identity key that can be used to link the records unequivocally, standard statistical techniques may produce misleading inference if the linked data are treated as if they were true observations. In this paper, we propose methods for categorical data analysis based on linked data that are not prepared by the analyst, such that neither the match-key variables nor the unlinked records are available. The adjustment is based on the proportion of false links in the linked file and our approach allows the probabilities of correct linkage to vary across the records without requiring that one is able to estimate this probability for each individual record. It accommodates also the general situation where unmatched records that cannot possibly be correctly linked exist in all the sources. The proposed methods are studied by simulation and applied to real data., (© 2024 The Author(s). Statistics in Medicine published by John Wiley & Sons Ltd.)

In evaluating the performance of different facilities or centers on survival outcomes, the standardized mortality ratio (SMR), which compares the observed to expected mortality has been widely used, particularly in the evaluation of kidney transplant centers. Despite its utility, the SMR may exaggerate center effects in settings where survival probability is relatively high. An example is one-year graft survival among U.S. kidney transplant recipients. We propose a novel approach to estimate center effects in terms of differences in survival probability (ie, each center versus a reference population). An essential component of the method is a prognostic score weighting technique, which permits accurately evaluating centers without necessarily specifying a correct survival model. Advantages of our approach over existing facility-profiling methods include a metric based on survival probability (greater clinical relevance than ratios of counts/rates); direct standardization (valid to compare between centers, unlike indirect standardization based methods, such as the SMR); and less reliance on correct model specification (since the assumed model is used to generate risk classes as opposed to fitted-value based 'expected' counts). We establish the asymptotic properties of the proposed weighted estimator and evaluate its finite-sample performance under a diverse set of simulation settings. The method is then applied to evaluate U.S. kidney transplant centers with respect to graft survival probability., (© 2024 John Wiley & Sons Ltd.)

In randomized trials, adjustment for measured covariates during the analysis can reduce variance and increase power. To avoid misleading inference, the analysis plan must be pre-specified. However, it is often unclear a priori which baseline covariates (if any) should be adjusted for in the analysis. Consider, for example, the Sustainable East Africa Research in Community Health (SEARCH) trial for HIV prevention and treatment. There are 16 matched pairs of communities and many potential adjustment variables, including region, HIV prevalence, male circumcision coverage, and measures of community-level viral load. In this paper, we propose a rigorous procedure to data-adaptively select the adjustment set, which maximizes the efficiency of the analysis. Specifically, we use cross-validation to select from a pre-specified library the candidate targeted maximum likelihood estimator (TMLE) that minimizes the estimated variance. For further gains in precision, we also propose a collaborative procedure for estimating the known exposure mechanism. Our small sample simulations demonstrate the promise of the methodology to maximize study power, while maintaining nominal confidence interval coverage. We show how our procedure can be tailored to the scientific question (intervention effect for the study sample vs. for the target population) and study design (pair-matched or not). Copyright © 2016 John Wiley & Sons, Ltd.

Introduction: There is currently no guidance on how to assess the calibration of multistate models used for risk prediction. We introduce several techniques that can be used to produce calibration plots for the transition probabilities of a multistate model, before assessing their performance in the presence of random and independent censoring through a simulation., Methods: We studied pseudo-values based on the Aalen-Johansen estimator, binary logistic regression with inverse probability of censoring weights (BLR-IPCW), and multinomial logistic regression with inverse probability of censoring weights (MLR-IPCW). The MLR-IPCW approach results in a calibration scatter plot, providing extra insight about the calibration. We simulated data with varying levels of censoring and evaluated the ability of each method to estimate the calibration curve for a set of predicted transition probabilities. We also developed evaluated the calibration of a model predicting the incidence of cardiovascular disease, type 2 diabetes and chronic kidney disease among a cohort of patients derived from linked primary and secondary healthcare records., Results: The pseudo-value, BLR-IPCW, and MLR-IPCW approaches give unbiased estimates of the calibration curves under random censoring. These methods remained predominately unbiased in the presence of independent censoring, even if the censoring mechanism was strongly associated with the outcome, with bias concentrated in low-density regions of predicted transition probability., Conclusions: We recommend implementing either the pseudo-value or BLR-IPCW approaches to produce a calibration curve, combined with the MLR-IPCW approach to produce a calibration scatter plot. The methods have been incorporated into the "calibmsm" R package available on CRAN., (© 2024 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Propensity score methods, such as inverse probability-of-treatment weighting (IPTW), have been increasingly used for covariate balancing in both observational studies and randomized trials, allowing the control of both systematic and chance imbalances. Approaches using IPTW are based on two steps: (i) estimation of the individual propensity scores (PS), and (ii) estimation of the treatment effect by applying PS weights. Thus, a variance estimator that accounts for both steps is crucial for correct inference. Using a variance estimator which ignores the first step leads to overestimated variance when the estimand is the average treatment effect (ATE), and to under or overestimated estimates when targeting the average treatment effect on the treated (ATT). In this article, we emphasize the importance of using an IPTW variance estimator that correctly considers the uncertainty in PS estimation. We present a comprehensive tutorial to obtain unbiased variance estimates, by proposing and applying a unifying formula for different types of PS weights (ATE, ATT, matching and overlap weights). This can be derived either via the linearization approach or M-estimation. Extensive R code is provided along with the corresponding large-sample theory. We perform simulation studies to illustrate the behavior of the estimators under different treatment and outcome prevalences and demonstrate appropriate behavior of the analytical variance estimator. We also use a reproducible analysis of observational lung cancer data as an illustrative example, estimating the effect of receiving a PET-CT scan on the receipt of surgery., (© 2024 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

The expected value of the standard power function of a test, computed with respect to a design prior distribution, is often used to evaluate the probability of success of an experiment. However, looking only at the expected value might be reductive. Instead, the whole probability distribution of the power function induced by the design prior can be exploited. In this article we consider one-sided testing for the scale parameter of exponential families and we derive general unifying expressions for cumulative distribution and density functions of the random power. Sample size determination criteria based on alternative summaries of these functions are discussed. The study sheds light on the relevance of the choice of the design prior in order to construct a successful experiment., (© 2024 John Wiley & Sons Ltd.)

The self-controlled case series (SCCS) is a commonly adopted study design in the assessment of vaccine and drug safety. Recurrent event data collected from SCCS studies are typically analyzed using the conditional Poisson model which assumes event times are independent within-cases. This assumption is violated in the presence of event dependence, where the occurrence of an event influences the probability and timing of subsequent events. When event dependence is suspected in an SCCS study, the standard recommendation is to include only the first event from each case in the analysis. However, first event analysis can still yield biased estimates of the exposure relative incidence if the outcome event is not rare. We first demonstrate that the bias in first event analysis can be even higher than previously assumed when subpopulations with different baseline incidence rates are present and describe an improved method for estimating this bias. Subsequently, we propose a novel partitioned analysis method and demonstrate how it can reduce this bias. We provide a recommendation to guide the number of partitions to use with the partitioned analysis, illustrate this recommendation with an example SCCS study of the association between beta-blockers and acute myocardial infarction, and compare the partitioned analysis against other SCCS analysis methods by simulation., (© 2024 John Wiley & Sons Ltd.)

Current status data are a type of failure time data that arise when the failure time of study subject cannot be determined precisely but is known only to occur before or after a random monitoring time. Variable selection methods for the failure time data have been discussed extensively in the literature. However, the statistical inference of the model selected based on the variable selection method ignores the uncertainty caused by model selection. To enhance the prediction accuracy for risk quantities such as survival probability, we propose two optimal model averaging methods under semiparametric additive hazards models. Specifically, based on martingale residuals processes, a delete-one cross-validation (CV) process is defined, and two new CV functional criteria are derived for choosing model weights. Furthermore, we present a greedy algorithm for the implementation of the techniques, and the asymptotic optimality of the proposed model averaging approaches is established, along with the convergence of the greedy averaging algorithms. A series of simulation experiments demonstrate the effectiveness and superiority of the proposed methods. Finally, a real-data example is provided as an illustration., (© 2024 John Wiley & Sons Ltd.)

We consider estimation of the semiparametric additive hazards model with an unspecified baseline hazard function where the effect of a continuous covariate has a specific shape but otherwise unspecified. Such estimation is particularly useful for a unimodal hazard function, where the hazard is monotone increasing and monotone decreasing with an unknown mode. A popular approach of the proportional hazards model is limited in such setting due to the complicated structure of the partial likelihood. Our model defines a quadratic loss function, and its simple structure allows a global Hessian matrix that does not involve parameters. Thus, once the global Hessian matrix is computed, a standard quadratic programming method can be applicable by profiling all possible locations of the mode. However, the quadratic programming method may be inefficient to handle a large global Hessian matrix in the profiling algorithm due to a large dimensionality, where the dimension of the global Hessian matrix and number of hypothetical modes are the same order as the sample size. We propose the quadratic pool adjacent violators algorithm to reduce computational costs. The proposed algorithm is extended to the model with a time-dependent covariate with monotone or U-shape hazard function. In simulation studies, our proposed method improves computational speed compared to the quadratic programming method, with bias and mean square error reductions. We analyze data from a recent cardiovascular study., (© 2024 John Wiley & Sons Ltd.)

Missing data in covariates can result in biased estimates and loss of power to detect associations. We consider Cox regression in which some covariates are subject to missing. The inverse probability weighted approach is often applied to regression analysis with missing covariates. Inverse probability weighted estimators typically are less efficient than likelihood-based estimators, but in general are more robust against model misspecification. In this article, we propose a robust best linear weighted estimator for Cox regression with missing covariates. Our proposed estimator is the projection of the simple inverse probability weighted estimator onto the orthogonal complement of the score space based on a working regression model of the observed data. The efficiency gain is from the use of the association between the survival outcome variable and the available covariates, which is the working regression model. The asymptotic distribution is derived, and the finite sample performance of the proposed estimator is examined via extensive simulation studies. The methods are applied to a colorectal cancer study to assess the association of the microsatellite instability status with colorectal cancer-specific mortality., (© 2024 John Wiley & Sons Ltd.)

As one of the most commonly used data types, methods in testing or designing a trial for binary endpoints from two independent populations are still being developed until recently. However, the power and the minimum required sample size comparisons between different tests may not be valid if their type I errors are not controlled at the same level. In this article, we unify all related testing procedures into a decision framework, including both frequentist and Bayesian methods. Sufficient conditions of the type I error attained at the boundary of hypotheses are derived, which help reduce the magnitude of the exact calculations and lay out a foundation for developing computational algorithms to correctly specify the actual type I error. The efficient algorithms are thus proposed to calculate the cutoff value in a deterministic decision rule and the probability value in a randomized decision rule, such that the actual type I error is under but closest to, or equal to, the intended level, respectively. The algorithm may also be used to calculate the sample size to achieve the prespecified type I error and power. The usefulness of the proposed methodology is further demonstrated in the power calculation for designing superiority and noninferiority trials., (© 2024 John Wiley & Sons Ltd.)

Accurate discrimination has been the central goal in identifying biomarkers for monitoring disease progression and early detection. Acknowledging the fact that discrimination accuracy of biomarkers for a time-to-event outcome often changes over time, local measures such as the time-dependent receiver operating characteristic curve and its area under the curve (AUC) are used to assess time-dependent predictive discrimination. However, such measures do not address subject heterogeneity, although the impact of covariates including demographics, disease-related characteristics, and other clinical information on the discriminatory performance of biomarkers needs to be investigated before their clinical use. We propose the covariate-specific time-dependent AUC, a measure for covariate-adjusted discrimination. We develop a regression model on the covariate-specific time-dependent AUC to understand how and in what magnitude the covariates influence biomarker performance. Then we construct a pseudo partial-likelihood for estimation and inference. This is followed by our establishing the asymptotic properties of the proposed estimators and provide variance estimation. The simulation studies and application to the AIDS Clinical Trials Group 175 data demonstrate that the proposed method offers an informative tool for inferring covariate-specific and time-dependent predictive discrimination., (© 2024 John Wiley & Sons, Ltd.)

In clinical studies, multi-state model (MSM) analysis is often used to describe the sequence of events that patients experience, enabling better understanding of disease progression. A complicating factor in many MSM studies is that the exact event times may not be known. Motivated by a real dataset of patients who received stem cell transplants, we considered the setting in which some event times were exactly observed and some were missing. In our setting, there was little information about the time intervals in which the missing event times occurred and missingness depended on the event type, given the analysis model covariates. These additional challenges limited the usefulness of some missing data methods (maximum likelihood, complete case analysis, and inverse probability weighting). We show that multiple imputation (MI) of event times can perform well in this setting. MI is a flexible method that can be used with any complete data analysis model. Through an extensive simulation study, we show that MI by predictive mean matching (PMM), in which sampling is from a set of observed times without reliance on a specific parametric distribution, has little bias when event times are missing at random, conditional on the observed data. Applying PMM separately for each sub-group of patients with a different pathway through the MSM tends to further reduce bias and improve precision. We recommend MI using PMM methods when performing MSM analysis with Markov models and partially observed event times., (© 2024 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Allocating patients to treatment arms during a trial based on the observed responses accumulated up to the decision point, and sequential adaptation of this allocation, could minimize the expected number of failures or maximize total benefits to patients. In this study, we developed a Bayesian response-adaptive randomization (RAR) design targeting the endpoint of organ support-free days (OSFD) for patients admitted to the intensive care units. The OSFD is a mixture of mortality and morbidity assessed by the number of days of free of organ support within a predetermined post-randomization time-window. In the past, researchers treated OSFD as an ordinal outcome variable where the lowest category is death. We propose a novel RAR design for a composite endpoint of mortality and morbidity, for example, OSFD, by using a Bayesian mixture model with a Markov chain Monte Carlo sampling to estimate the posterior probability distribution of OSFD and determine treatment allocation ratios at each interim. Simulations were conducted to compare the performance of our proposed design under various randomization rules and different alpha spending functions. The results show that our RAR design using Bayesian inference allocated more patients to the better performing arm(s) compared to other existing adaptive rules while assuring adequate power and type I error rate control across a range of plausible clinical scenarios., (© 2024 John Wiley & Sons Ltd.)

In recent decades, several randomization designs have been proposed in the literature as better alternatives to the traditional permuted block design (PBD), providing higher allocation randomness under the same restriction of the maximum tolerated imbalance (MTI). However, PBD remains the most frequently used method for randomizing subjects in clinical trials. This status quo may reflect an inadequate awareness and appreciation of the statistical properties of these randomization designs, and a lack of simple methods for their implementation. This manuscript presents the analytic results of statistical properties for five randomization designs with MTI restriction based on their steady-state probabilities of the treatment imbalance Markov chain and compares them to those of the PBD. A unified framework for randomization sequence generation and real-time on-demand treatment assignment is proposed for the straightforward implementation of randomization algorithms with explicit formulas of conditional allocation probabilities. Topics associated with the evaluation, selection, and implementation of randomization designs are discussed. It is concluded that for two-arm equal allocation trials, several randomization designs offer stronger protection against selection bias than the PBD does, and their implementation is not necessarily more difficult than the implementation of the PBD., (© 2024 John Wiley & Sons Ltd.)

State-of-the-art biostatistics methods allow for the simultaneous modeling of several correlated non-fatal disease processes over time, but there is no clear guidance on the optimal analysis in most settings. An example occurs in diabetes, where it is not known with certainty how microvascular complications of the eyes, kidneys, and nerves co-develop over time. In this article, we propose and contrast two general model frameworks for studying complications (sequential state and parallel trajectory frameworks) and review multivariate methods for their analysis, focusing on multistate and joint modeling. We illustrate these methods in a tutorial format using the long-term follow-up from the Diabetes Control and Complications Trial and Epidemiology of Diabetes Interventions and Complications study public data repository. A formal comparison of prediction error and discrimination is included. Multistate models are particularly advantageous for determining the order and timing of complications, but require discretization of the longitudinal outcomes and possibly a very complex state space process. Intermittent observation of the states must be accounted for, and discretization is a probable disadvantage in this setting. In contrast, joint models can account for variations of continuous biomarkers over time and are particularly designed for modeling complex association structures between the complications and for performing dynamic predictions of an outcome of interest to inform clinical decisions (eg, a late-stage complication). We found that both models have helpful features that can better-inform our understanding of the complex trajectories that complications may take and can therefore help with decision making for patients presenting with diabetes complications., (© 2023 John Wiley & Sons Ltd.)

The population-attributable fraction (PAF) is commonly interpreted as the proportion of events that can be ascribed to a certain exposure in a certain population. Its estimation is sensitive to common forms of time-dependent bias in the face of a time-dependent exposure. Predominant estimation approaches based on multistate modeling fail to fully eliminate such bias and, as a result, do not permit a causal interpretation, even in the absence of confounding. While recently proposed multistate modeling approaches can successfully eliminate residual time-dependent bias, and moreover succeed to adjust for time-dependent confounding by means of inverse probability of censoring weighting, inadequate application, and misinterpretation prevails in the medical literature. In this paper, we therefore revisit recent work on previously proposed PAF estimands and estimators in settings with time-dependent exposures and competing events and extend this work in several ways. First, we critically revisit the interpretation and applied terminology of these estimands. Second, we further formalize the assumptions under which a causally interpretable PAF estimand can be identified and provide analogous weighting-based representations of the identifying functionals of other proposed estimands. This representation aims to enhance the applied statistician's understanding of different sources of bias that may arise when the aim is to obtain a valid estimate of a causally interpretable PAF. To illustrate and compare these representations, we present a real-life application to observational data from the Ghent University Hospital ICUs to estimate the fraction of ICU deaths attributable to hospital-acquired infections., (© 2023 John Wiley & Sons Ltd.)

Multiple mediation analysis is a powerful methodology to assess causal effects in the presence of multiple mediators. Several methodologies, such as G-computation and inverse-probability-weighting, have been widely used to draw inferences about natural indirect effects (NIEs). However, a limitation of these methods is their potential for model misspecification. Although powerful semiparametric methods with high robustness and consistency have been developed for inferring average causal effects and for analyzing the effects of a single mediator, a comparably robust method for multiple mediation analysis is still lacking. Therefore, this theoretical study proposes a method of using multiply robust estimators of NIEs in the presence of multiple ordered mediators. We show that the proposed estimators not only enjoy the multiply robustness to model misspecification, they are also consistent and asymptotically normal under regular conditions. We also performed simulations for empirical comparisons of the finite-sample properties between our multiply robust estimators and existing methods. In an illustrative example, a dataset for liver disease patients in Taiwan is used to examine the mediating roles of liver damage and liver cancer in the pathway from hepatitis B/C virus infection to mortality. The model is implemented in the open-source R package "MedMR.", (© 2023 John Wiley & Sons Ltd.)

There are now many options for doubly robust estimation; however, there is a concerning trend in the applied literature to believe that the combination of a propensity score and an adjusted outcome model automatically results in a doubly robust estimator and/or to misuse more complex established doubly robust estimators. A simple alternative, canonical link generalized linear models (GLM) fit via inverse probability of treatment (propensity score) weighted maximum likelihood estimation followed by standardization (the g $$ g $$ -formula) for the average causal effect, is a doubly robust estimation method. Our aim is for the reader not just to be able to use this method, which we refer to as IPTW GLM, for doubly robust estimation, but to fully understand why it has the doubly robust property. For this reason, we define clearly, and in multiple ways, all concepts needed to understand the method and why it is doubly robust. In addition, we want to make very clear that the mere combination of propensity score weighting and an adjusted outcome model does not generally result in a doubly robust estimator. Finally, we hope to dispel the misconception that one can adjust for residual confounding remaining after propensity score weighting by adjusting in the outcome model for what remains 'unbalanced' even when using doubly robust estimators. We provide R code for our simulations and real open-source data examples that can be followed step-by-step to use and hopefully understand the IPTW GLM method. We also compare to a much better-known but still simple doubly robust estimator., (© 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Incorporating interim analysis into a trial design is gaining popularity in the field of confirmatory clinical trials, where two studies may be conducted in parallel (ie, twin studies) in order to provide substantial evidence per the requirement of FDA guidance. Interim futility analysis provides a chance to check for the "disaster" scenario when the treatment has a high probability to be not more efficacious than the control. Therefore, it is an efficient tool to mitigate risk of running a complete and expansive trial under such scenario. There is no agreement among trial designers that interim analysis should be based on individual study data or pooled data under the twin study scenario. In fact, it is a dilemma for most scientists when specifying the interim analysis strategy at the design stage as the true treatment effects of the twin studies are unknown no matter how similar they are intended to be. To address the issue, we developed a Bayesian hierarchical modeling method to allow dynamic data borrowing between twin studies and demonstrated a favorable characteristic of the new method over the separate and pooled analyses. We evaluated a wide spectrum of the heterogeneity hyperparameters and visualized its critical impact on the Bayesian model's characteristic. Based on the evaluation, we made a suggestion on the heterogeneity hyperparameter selection independent of any a priori knowledge. We also applied our method to a case study where predictive powers of different methods are compared., (© 2023 John Wiley & Sons Ltd.)

Regulatory guidelines mandate the strong control of the familywise error rate in confirmatory clinical trials with primary and secondary objectives. Bonferroni tests are one of the popular choices for multiple comparison procedures and are building blocks of more advanced procedures. It is usually of interest to find the optimal weighted Bonferroni split for multiple hypotheses. We consider two popular quantities as the optimization objectives, which are the disjunctive power and the conjunctive power. The former is the probability to reject at least one false hypothesis and the latter is the probability to reject all false hypotheses. We investigate the behavior of each of them as a function of different Bonferroni splits, given assumptions about the alternative hypotheses and correlations between test statistics. Under independent tests, unique optimal Bonferroni weights exist; under dependence, optimal Bonferroni weights may not be unique based on a fine grid search. In general, we propose an optimization algorithm based on constrained nonlinear optimization and multiple starting points. The proposed algorithm efficiently identifies optimal Bonferroni weights to maximize the disjunctive or conjunctive power. In addition, we apply the proposed algorithm to graphical approaches, which include many Bonferroni-based multiple comparison procedures. Utilizing the closed testing principle, we adopt a two-step approach to find optimal graphs using the disjunctive power. We also identify a class of closed test procedures that optimize the conjunctive power. We apply the proposed algorithm to a case study to illustrate the utility of optimal graphical approaches that reflect study objectives., (© 2023 John Wiley & Sons Ltd.)

The use of Monte-Carlo (MC) p $$ p $$ -values when testing the significance of a large number of hypotheses is now commonplace. In large-scale hypothesis testing, we will typically encounter at least some p $$ p $$ -values near the threshold of significance, which require a larger number of MC replicates than p $$ p $$ -values that are far from the threshold. As a result, some incorrect conclusions can be reached due to MC error alone; for hypotheses near the threshold, even a very large number (eg, 1 0 6 $$ 1{0}^6 $$ ) of MC replicates may not be enough to guarantee conclusions reached using MC p $$ p $$ -values. Gandy and Hahn (GH) 6-8 have developed the only method that directly addresses this problem. They defined a Monte-Carlo error rate (MCER) to be the probability that any decisions on accepting or rejecting a hypothesis based on MC p $$ p $$ -values are different from decisions based on ideal p $$ p $$ -values; their method then makes decisions by controlling the MCER. Unfortunately, the GH method is frequently very conservative, often making no rejections at all and leaving a large number of hypotheses "undecided". In this article, we propose MERIT, a method for large-scale MC hypothesis testing that also controls the MCER but is more statistically efficient than the GH method. Through extensive simulation studies, we demonstrate that MERIT controls the MCER while making more decisions that agree with the ideal p $$ p $$ -values than GH does. We also illustrate our method by an analysis of gene expression data from a prostate cancer study., (© 2023 John Wiley & Sons, Ltd.)

Postmarket safety surveillance is an integral part of mass vaccination programs. Typically relying on sequential analysis of real-world health data as they accrue, safety surveillance is challenged by sequential multiple testing and by biases induced by residual confounding in observational data. The current standard approach based on the maximized sequential probability ratio test (MaxSPRT) fails to satisfactorily address these practical challenges and it remains a rigid framework that requires prespecification of the surveillance schedule. We develop an alternative Bayesian surveillance procedure that addresses both aforementioned challenges using a more flexible framework. To mitigate bias, we jointly analyze a large set of negative control outcomes that are adverse events with no known association with the vaccines in order to inform an empirical bias distribution, which we then incorporate into estimating the effect of vaccine exposure on the adverse event of interest through a Bayesian hierarchical model. To address multiple testing and improve on flexibility, at each analysis timepoint, we update a posterior probability in favor of the alternative hypothesis that vaccination induces higher risks of adverse events, and then use it for sequential detection of safety signals. Through an empirical evaluation using six US observational healthcare databases covering more than 360 million patients, we benchmark the proposed procedure against MaxSPRT on testing errors and estimation accuracy, under two epidemiological designs, the historical comparator and the self-controlled case series. We demonstrate that our procedure substantially reduces Type 1 error rates, maintains high statistical power and fast signal detection, and provides considerably more accurate estimation than MaxSPRT. Given the extensiveness of the empirical study which yields more than 7 million sets of results, we present all results in a public R ShinyApp. As an effort to promote open science, we provide full implementation of our method in the open-source R package EvidenceSynthesis., (© 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Stepped-wedge cluster randomized trials (SW-CRTs) are typically analyzed assuming a constant intervention effect. In practice, the intervention effect may vary as a function of exposure time, leading to biased results. The estimation of time-on-intervention (TOI) effects specifies separate discrete intervention effects for each elapsed period of exposure time since the intervention was first introduced. It has been demonstrated to produce results with minimum bias and nominal coverage probabilities in the analysis of SW-CRTs. Due to the design's staggered crossover, TOI effect variances are heteroskedastic in a SW-CRT. Accordingly, we hypothesize that alternative CRT designs will be more efficient at modeling certain TOI effects. We derive and compare the variance estimators of TOI effects between a SW-CRT, parallel CRT (P-CRT), parallel CRT with baseline (PB-CRT), and novel parallel CRT with baseline and an all-exposed period (PBAE-CRT). We also prove that the time-averaged TOI effect variance and point estimators are identical to that of the constant intervention effect in both P-CRTs and PB-CRTs. We then use data collected from a hospital disinvestment study to simulate and compare the differences in TOI effect estimates between the different CRT designs. Our results reveal that the SW-CRT has the most efficient estimator for the early TOI effect, whereas the PB-CRT typically has the most efficient estimator for the long-term and time-averaged TOI effects. Overall, the PB-CRT with TOI effects can be a more appropriate choice of CRT design for modeling intervention effects that vary by exposure time., (© 2023 John Wiley & Sons Ltd.)

The win ratio has become a popular method for comparing multiple event data between two groups in clinical cohort studies. The win ratio compares the event data in prioritized order, where the first prioritized event is death and a typical example for the second prioritized event is hospitalization. Literature is sparse on inference for win and loss parameters, including the win ratio, for censored event data. Inference for two prioritized censored event times has been developed for independent right-censoring. Many clinical studies include recurrent event data such as hospitalizations. In this article, we suggest inference for win-loss parameters for death and a recurrent event outcome under independent right-censoring. The small sample properties of the proposed method are studied in a simulation study showing that the variance formula is accurate even for small samples. The method is applied on a data set from a randomized clinical trial., (© 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Imputation of longitudinal categorical covariates with several waves and many predictors is cumbersome in terms of implausible transitions, colinearity, and overfitting. We designed a simulation study with data obtained from a general practitioners' morbidity registry in Belgium for three waves, with smoking as the longitudinal covariate of interest. We set varying proportions of data on smoking to missing completely at random and missing not at random with proportions of missingness equal to 10%, 30%, 50%, and 70%. This study proposed a 3-stage approach that allows flexibility when imputing time-dependent categorical covariates. First, multiple imputation using fully conditional specification or multiple imputation for the predictor variables was deployed using the wide format such that previous and future information of the same patient was utilized. Second, a joint Markov transition model for initial, forward, backward, and intermittent probabilities was developed for each imputed dataset. Finally, this transition model was used for imputation. We compared the performance of this methodology with an analyses of the complete data and with listwise deletion in terms of bias and root mean square error. Next, we applied this methodology in a clinical case for years 2017 to 2021, where we estimated the effect of several covariates on the pneumococcal vaccination. This methodological framework ensures that the plausibility of transitions is preserved, overfitting and colinearity issues are resolved, and confounders can be utilized. Finally, a companion R package was developed to enable the replication and easy application of this methodology., (© 2023 John Wiley & Sons Ltd.)

Statistical prediction models have gained popularity in applied research. One challenge is the transfer of the prediction model to a different population which may be structurally different from the model for which it has been developed. An adaptation to the new population can be achieved by calibrating the model to the characteristics of the target population, for which numerous calibration techniques exist. In view of this diversity, we performed a systematic evaluation of various popular calibration approaches used by the statistical and the machine learning communities for estimating two-class probabilities. In this work, we first provide a review of the literature and, second, present the results of a comprehensive simulation study. The calibration approaches are compared with respect to their empirical properties and relationships, their ability to generalize precise probability estimates to external populations and their availability in terms of easy-to-use software implementations. Third, we provide code from real data analysis allowing its application by researchers. Logistic calibration and beta calibration, which estimate an intercept plus one and two slope parameters, respectively, consistently showed the best results in the simulation studies. Calibration on logit transformed probability estimates generally outperformed calibration methods on nontransformed estimates. In case of structural differences between training and validation data, re-estimation of the entire prediction model should be outweighted against sample size of the validation data. We recommend regression-based calibration approaches using transformed probability estimates, where at least one slope is estimated in addition to an intercept for updating probability estimates in validation studies., (© 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

Graphical approach provides a useful framework for multiplicity adjustment in clinical trials with multiple endpoints. When designing a graphical approach, initial weight and transition probability for the endpoints are often assigned based on clinical importance. For example, practitioners may prefer putting more weights on some primary endpoints. The clinical preference can be formulated as a constrain in the sample size optimization problem. However, there has been a lack of theoretical guidance on how to specify initial weight and transition probability in a graphical approach to meet the clinical preference but at the same time to minimize the sample size needed for a power requirement. To fill this gap, we propose statistical methods to optimize sample size over initial weight and transition probability in a graphical approach under a common setting, which is to use marginal power for each endpoint in a trial design. Importantly, we prove that some of the commonly used graphical approaches such as putting all initial weights on one endpoint are suboptimal. Our methods are flexible, which can be used for both single-arm trials and randomized controlled trials with either continuous or binary or mixed types of endpoints. Additionally, we prove the existence of optimal solution where all marginal powers are placed exactly at the prespecified values, assuming continuity. Two hypothetical clinical trial designs are presented to illustrate the application of our methods under different scenarios. Results are first presented for a design with two endpoints and are further generalized to three or more endpoints. Our findings are helpful to guide the design of a graphical approach and the sample size calculation in clinical trials., (© 2023 John Wiley & Sons Ltd.)

The medical field commonly employs post-test measures such as predictive values and likelihood ratios to assess diagnostic accuracy. Predictive values, including positive and negative values (PPV and NPV), indicate the probability that individuals have a target health condition based on test results. On the other hand, likelihood ratios, including positive and negative ratios (LR+ and LR- respectively), compare the probability of a particular test result between the diseased and non-diseased groups. While predictive values are useful in evaluating diagnostic test accuracy in populations with varying disease prevalence, likelihood ratios provide a direct link between pre-test and post-test probabilities in specific patients. In this study, we introduce and analyze a new approach called generalized predictive values and likelihood ratios, using a tree ordering of disease classes. We evaluate the effectiveness of these methods through simulation studies and illustrate their use with real data on lung cancer., (© 2023 John Wiley & Sons Ltd.)

Comparative effectiveness research is often concerned with evaluating treatment strategies sustained over time, that is, time-varying treatments. Inverse probability weighting (IPW) is often used to address the time-varying confounding by re-weighting the sample according to the probability of treatment receipt at each time point. IPW can also be used to address any missing data by re-weighting individuals according to the probability of observing the data. The combination of these two distinct sets of weights may lead to inefficient estimates of treatment effects due to potentially highly variable total weights. Alternatively, multiple imputation (MI) can be used to address the missing data by replacing each missing observation with a set of plausible values drawn from the posterior predictive distribution of the missing data given the observed data. Recent studies have compared IPW and MI for addressing the missing data in the evaluation of time-varying treatments, but they focused on missing confounders and monotone missing data patterns. This article assesses the relative advantages of MI and IPW to address missing data in both outcomes and confounders measured over time, and across monotone and non-monotone missing data settings. Through a comprehensive simulation study, we find that MI consistently provided low bias and more precise estimates compared to IPW across a wide range of scenarios. We illustrate the implications of method choice in an evaluation of biologic drugs for patients with severe rheumatoid arthritis, using the US National Databank for Rheumatic Diseases, in which 25% of participants had missing health outcomes or time-varying confounders., (© 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

In immuno-oncology clinical trials, multiple immunological biomarkers are usually examined over time to comprehensively and appropriately evaluate the efficacy of treatments. Because predicting patients' future survival statuses on the basis of such recorded longitudinal information might be of great interest, joint modeling of longitudinal and time-to-event data has been intensively discussed as a toolkit to implement such a prediction. To achieve a desirable predictive performance, averaging over multiple candidate predictive models to account for the model uncertainty might be a more suitable statistical approach than selecting the single best model. Although Bayesian model averaging can be one of the approaches, several problems related to model weights with marginal likelihoods have been discussed. To address these problems, we here propose a Bayesian predictive model averaging (BPMA) method that uses Bayesian leave-one-out cross-validation predictive densities to account for the subject-specific and time-dependent nature of the prediction. We examine the operating characteristics of the proposed BPMA method in terms of the predictive accuracy (ie, the calibration and discrimination abilities) in extensive simulation studies. In addition, we discuss the strengths and limitations of the proposed method by applying it to an immuno-oncology clinical trial in patients with advanced ovarian cancer., (© 2023 John Wiley & Sons Ltd.)

Response-dependent sampling is routinely used as an enrichment strategy in the design of family studies investigating the heritable nature of disease. In addition to the response of primary interest, investigators often wish to investigate the association between biomarkers and secondary responses related to possible comorbidities. Statistical analysis regarding genetic biomarkers and their association with the secondary outcome must address the biased sampling scheme involving the primary response. In this article, we develop composite likelihoods and two-stage estimation procedures for such secondary analyses in which the within-family dependence structure for the primary and secondary outcomes is modeled via a Gaussian copula. The dependence among responses within family members is modeled based on kinship coefficients. Auxiliary data from independent individuals are exploited by augmenting the composite likelihoods to increase precision of marginal parameter estimates and enhance the efficiency of estimators of the dependence parameters. Simulation studies are carried out to evaluate the finite sample performance of the proposed method, and an application to a motivating family study in psoriatic arthritis is given for illustration., (© 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)

The Glasgow outcome scale-extended (GOS-E), an ordinal scale measure, is often selected as the endpoint for clinical trials of traumatic brain injury (TBI). Traditionally, GOS-E is analyzed as a fixed dichotomy with favorable outcome defined as GOS-E ≥ 5 and unfavorable outcome as GOS-E < 5. More recent studies have defined favorable vs unfavorable outcome utilizing a sliding dichotomy of the GOS-E that defines a favorable outcome as better than a subject's predicted prognosis at baseline. Both dichotomous approaches result in loss of statistical and clinical information. To improve on power, Yeatts et al proposed a sliding scoring of the GOS-E as the distance from the cutoff for favorable/unfavorable outcomes, and therefore used more information found in the original GOS-E to estimate the probability of favorable outcome. We used data from a published TBI trial to explore the ramifications to trial operating characteristics by analyzing the sliding scoring of the GOS-E as either dichotomous, continuous, or ordinal. We illustrated a connection between the ordinal data and time-to-event (TTE) data to allow use of Bayesian software that utilizes TTE-based modeling. The simulation results showed that the continuous method with continuity correction offers higher power and lower mean squared error for estimating the probability of favorable outcome compared to the dichotomous method, and similar power but higher precision compared to the ordinal method. Therefore, we recommended that future severe TBI clinical trials consider analyzing the sliding scoring of the GOS-E endpoint as continuous with continuity correction., (© 2023 John Wiley & Sons Ltd.)

Medical cost data often consist of zero values as well as extremely right-skewed positive values. A two-part model is a popular choice for analyzing medical cost data, where the first part models the probability of a positive cost using logistic regression and the second part models the positive cost using a lognormal or Gamma distribution. To address the unmeasured confounding in studies on cost outcome under two-part models, two instrumental variable (IV) methods, two-stage residual inclusion (2SRI) and two-stage prediction substitution (2SPS) are widely applied. However, previous literature demonstrated that both the 2SRI and the 2SPS could fail to consistently estimate the causal effect among compliers under standard IV assumptions for binary and survival outcomes. Our simulation studies confirmed that it continued to be the case for a two-part model, which is another nonlinear model. In this article, we develop a model-based IV approach, Instrumental Variable with Two-Part model (IV2P), to obtain a consistent estimate of the causal effect among compliers for cost outcome under standard IV assumptions. In addition, we develop sensitivity analysis approaches to allow the evaluation of the sensitivity of the causal conclusions to potential quantified violations of the exclusion restriction assumption and the randomization of IV assumption. We apply our method to a randomized cash incentive study to evaluate the effect of a primary care visit on medical cost among low-income adults newly covered by a primary care program., (© 2023 John Wiley & Sons Ltd.)

Inverse probability weighting can be used to correct for missing data. New estimators for the weights in the nonmonotone setting were introduced in 2018. These estimators are the unconstrained maximum likelihood estimator (UMLE) and the constrained Bayesian estimator (CBE), an alternative if UMLE fails to converge. In this work we describe and illustrate these estimators, and examine performance in simulation and in an applied example estimating the effect of anemia on spontaneous preterm birth in the Zambia Preterm Birth Prevention Study. We compare performance with multiple imputation (MI) and focus on the setting of an observational study where inverse probability of treatment weights are used to address confounding. In simulation, weighting was less statistically efficient at the smallest sample size and lowest exposure prevalence examined (n = 1500, 15% respectively) but in other scenarios statistical performance of weighting and MI was similar. Weighting had improved computational efficiency taking, on average, 0.4 and 0.05 times the time for MI in R and SAS, respectively. UMLE was easy to implement in commonly used software and convergence failure occurred just twice in >200 000 simulated cohorts making implementation of CBE unnecessary. In conclusion, weighting is an alternative to MI for nonmonotone missingness, though MI performed as well as or better in terms of bias and statistical efficiency. Weighting's superior computational efficiency may be preferred with large sample sizes or when using resampling algorithms. As validity of weighting and MI rely on correct specification of different models, both approaches could be implemented to check agreement of results., (© 2023 John Wiley & Sons Ltd.)

Evaluating the prognostic performance of candidate markers for future disease onset or progression is one of the major goals in medical research. A marker's prognostic performance refers to how well it separates patients at the high or low risk of a future disease state. Often the discriminative performance of a marker is affected by the patient characteristics (covariates). Time-dependent receiver operating characteristic (ROC) curves that ignore the informativeness of the covariates will lead to biased estimates of the accuracy parameters. We propose a time-dependent ROC curve that accounts for the informativeness of the covariates in the case of censored data. We propose inverse probability weighted (IPW) estimators for estimating the proposed accuracy parameters. We investigate the performance of the IPW estimators through simulation studies and real-life data analysis., (© 2023 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.)