Search

Longitudinal observational data on patients can be used to investigate causal effects of time-varying treatments on time-to-event outcomes. Several methods have been developed for estimating such effects by controlling for the time-dependent confounding that typically occurs. The most commonly used is marginal structural models (MSM) estimated using inverse probability of treatment weights (IPTW) (MSM-IPTW). An alternative, the sequential trials approach, is increasingly popular, and involves creating a sequence of "trials" from new time origins and comparing treatment initiators and non-initiators. Individuals are censored when they deviate from their treatment assignment at the start of each "trial" (initiator or noninitiator), which is accounted for using inverse probability of censoring weights. The analysis uses data combined across trials. We show that the sequential trials approach can estimate the parameters of a particular MSM. The causal estimand that we focus on is the marginal risk difference between the sustained treatment strategies of "always treat" vs "never treat." We compare how the sequential trials approach and MSM-IPTW estimate this estimand, and discuss their assumptions and how data are used differently. The performance of the two approaches is compared in a simulation study. The sequential trials approach, which tends to involve less extreme weights than MSM-IPTW, results in greater efficiency for estimating the marginal risk difference at most follow-up times, but this can, in certain scenarios, be reversed at later time points and relies on modelling assumptions. We apply the methods to longitudinal observational data from the UK Cystic Fibrosis Registry to estimate the effect of dornase alfa on survival.

We use information derived from over 40K trials in the Cochrane Collaboration database of systematic reviews (CDSR) to compute the replication probability, or predictive power of an experiment given its observed (two-sided) P$$ P $$-value. We find that an exact replication of a marginally significant result with P=.05$$ P=.05 $$ has less than 30% chance of again reaching significance. Moreover, the replication of a result with P=.005$$ P=.005 $$ still has only 50% chance of significance. We also compute the probability that the direction (sign) of the estimated effect is correct, which is closely related to the type S error of Gelman and Tuerlinckx. We find that if an estimated effect has P=.05$$ P=.05 $$, there is a 93% probability that its sign is correct. If P=.005$$ P=.005 $$, then that probability is 99%. Finally, we compute the required sample size for a replication study to achieve some specified power conditional on the p$$ p $$-value of the original study. We find that the replication of a result with P=.05$$ P=.05 $$ requires a sample size more than 16 times larger than the original study to achieve 80% power, while P=.005$$ P=.005 $$ requires at least 3.5 times larger sample size. These findings confirm that failure to replicate the statistical significance of a trial does not necessarily indicate that the original result was a fluke.

The restricted mean survival time (RMST) is a clinically meaningful summary measure in studies with survival outcomes. Statistical methods have been developed for regression analysis of RMST to investigate impacts of covariates on RMST, which is a useful alternative to the Cox regression analysis. However, existing methods for regression modeling of RMST are not applicable to left-truncated right-censored data that arise frequently in prevalent cohort studies, for which the sampling bias due to left truncation and informative censoring induced by the prevalent sampling scheme must be properly addressed. The pseudo-observation (PO) approach has been used in regression modeling of RMST for right-censored data and competing-risks data. For left-truncated right-censored data, we propose to directly model RMST as a function of baseline covariates based on POs under general censoring mechanisms. We adjust for the potential covariate-dependent censoring or dependent censoring by the inverse probability of censoring weighting method. We establish large sample properties of the proposed estimators and assess their finite sample performances by simulation studies under various scenarios. We apply the proposed methods to a prevalent cohort of women diagnosed with stage IV breast cancer identified from surveillance, epidemiology, and end results-medicare linked database.

Stratification is commonly employed in clinical trials to reduce the chance covariate imbalances and increase the precision of the treatment effect estimate. We propose a general framework for constructing the confidence interval (CI) for a difference or ratio effect parameter under stratified sampling by the method of variance estimates recovery (MOVER). We consider the additive variance and additive CI approaches for the difference, in which either the CI for the weighted difference, or the CI for the weighted effect in each group, or the variance for the weighted difference is calculated as the weighted sum of the corresponding stratum-specific statistics. The CI for the ratio is derived by the Fieller and log-ratio methods. The weights can be random quantities under the assumption of a constant effect across strata, but this assumption is not needed for fixed weights. These methods can be easily applied to different endpoints in that they require only the point estimate, CI, and variance estimate for the measure of interest in each group across strata. The methods are illustrated with two real examples. In one example, we derive the MOVER CIs for the risk difference and risk ratio for binary outcomes. In the other example, we compare the restricted mean survival time and milestone survival in stratified analysis of time-to-event outcomes. Simulations show that the proposed MOVER CIs generally outperform the standard large sample CIs, and that the additive CI approach performs slightly better than the additive variance approach., Comment: 13 pages

Time-dependent receiver operating characteristic curves are often used to evaluate the classification performance of continuous measures when considering time-to-event data. When one is interested in evaluating the predictive performance of multiple covariates, it is common to use the Cox proportional hazards model to obtain risk scores; however, previous work has shown that when the model is mis-specified, the estimand corresponding to the partial likelihood estimator depends on the censoring distribution. In this manuscript, we show that when the risk score model is mis-specified, the AUC will also depend on the censoring distribution, leading to either over- or under-estimation of the risk score's predictive performance. We propose the use of censoring-robust estimators to remove the dependence on the censoring distribution and provide empirical results supporting the use of censoring-robust risk scores.

In clinical trials, surrogate endpoints are useful when the endpoint of interest is difficult to measure or requires a long follow-up time. Current methodology for validating surrogate endpoints encounters challenges in the presence of collinearity between the treatment and surrogate endpoint, which is often present in clinical trials. The proposed methods adapt current methodology in the structural framework of path analysis to quantify the validity of a surrogate endpoint. The path analysis framework provides an improved interpretation of treatment effect. Through derivation and simulation we show the proposed path likelihood reduction factor (LRF P ), is less biased and more robust than current methodology in cases of collinearity between the treatment and surrogate endpoint, with notable improvement when surrogacy is weak or moderate. LRF P can be expanded to evaluate multiple correlated surrogate endpoints, which as shown through simulation, is also less biased and more robust than current methodology in the case of collinearity between the treatment and surrogate endpoint.

A commonly reported measure when interested in the survival of cancer patients is relative survival. Relative survival circumvents issues with inaccurate cause of death information by incorporating the expected mortality rates of cancer individuals from population lifetables of the general population. A summary of the cancer population prognosis can be obtained using the marginal relative survival. To explore differences between exposure groups, such as socioeconomic groups, the difference in marginal relative survival between exposed and unexposed can be obtained and under assumptions is interpreted as the average causal effect of exposure to survival. In a modeling context, this is usually estimated by applying regression standardization as the average of the individual-specific estimates after fitting a relative survival model. Regression standardization yields an estimator that consistently estimates the causal effect under standard causal inference assumptions and if the relative survival model is correctly specified. We extend inverse probability weighting (IPW) and doubly robust standardization methods in the relative survival framework as additional valuable tools for obtaining average causal effects when correct model specification might not hold for the relative survival model. IPW yields an unbiased estimate of the average causal effect if a correctly specified model has been fitted for the exposure (propensity score) whereas doubly robust standardization requires that at least one of the propensity score model or the relative survival model is correctly specified. An example using data on melanoma is provided and a simulation study is conducted to investigate how sensitive are the methods to model misspecification, including different ways for obtaining standard errors.

Dynamic treatment regimes (DTRs) include a sequence of treatment decision rules, in which treatment is adapted over time in response to the changes in an individual’s disease progression and health care history. In medical practice, nested test-and-treat strategies are common to improve cost-effectiveness. For example, for patients at risk of prostate cancer, only patients who have high prostate-specific antigen (PSA) need a biopsy, which is costly and invasive, to confirm the diagnosis and help determine the treatment if needed. A decision about treatment happens after the biopsy, and is thus nested within the decision of whether to do the test. However, current existing statistical methods are not able to accommodate such a naturally embedded property of the treatment decision within the test decision. Therefore, we developed a new statistical learning method, Step-adjusted Tree-based Reinforcement Learning, to evaluate DTRs within such a nested multi-stage dynamic decision framework using observational data. At each step within each stage, we combined the robust semi-parametric estimation via Augmented Inverse Probability Weighting with a tree-based reinforcement learning method to deal with the counterfactual optimization. The simulation studies demonstrated robust performance of the proposed methods under different scenarios. We further applied our method to evaluate the necessity of prostate biopsy and identify the optimal test-and-treat regimes for prostate cancer patients using data from the Johns Hopkins University prostate cancer active surveillance dataset.

Individual participant data (IPD) from multiple sources allows external validation of a prognostic model across multiple populations. Often this reveals poor calibration, potentially causing poor predictive performance in some populations. However, rather than discarding the model outright, it may be possible to modify the model to improve performance using recalibration techniques. We use IPD meta-analysis to identify the simplest method to achieve good model performance. We examine four options for recalibrating an existing time-to-event model across multiple populations: (i) shifting the baseline hazard by a constant, (ii) re-estimating the shape of the baseline hazard, (iii) adjusting the prognostic index as a whole, and (iv) adjusting individual predictor effects. For each strategy, IPD meta-analysis examines (heterogeneity in) model performance across populations. Additionally, the probability of achieving good performance in a new population can be calculated allowing ranking of recalibration methods. In an applied example, IPD meta-analysis reveals that the existing model had poor calibration in some populations, and large heterogeneity across populations. However, re-estimation of the intercept substantially improved the expected calibration in new populations, and reduced between-population heterogeneity. Comparing recalibration strategies showed that re-estimating both the magnitude and shape of the baseline hazard gave the highest predicted probability of good performance in a new population. In conclusion, IPD meta-analysis allows a prognostic model to be externally validated in multiple settings, and enables recalibration strategies to be compared and ranked to decide on the least aggressive recalibration strategy to achieve acceptable external model performance without discarding existing model information.

Recent methodological developments in causal mediation analysis have addressed several issues regarding multiple mediators. However, these developed methods differ in their definitions of causal parameters, assumptions for identification, and interpretations of causal effects, making it unclear which method ought to be selected when investigating a given causal effect. Thus, in this study, we construct an integrated framework, which unifies all existing methodologies, as a standard for mediation analysis with multiple mediators. To clarify the relationship between existing methods, we propose four strategies for effect decomposition: two-way, partially forward, partially backward, and complete decompositions. This study reveals how the direct and indirect effects of each strategy are explicitly and correctly interpreted as path-specific effects under different causal mediation structures. In the integrated framework, we further verify the utility of the interventional analogues of direct and indirect effects, especially when natural direct and indirect effects cannot be identified or when crossworld exchangeability is invalid. Consequently, this study yields a robustness-specificity trade-off in the choice of strategies. Inverse probability weighting is considered for estimation. The four strategies are further applied to a simulation study for performance evaluation and for analyzing the Risk Evaluation of Viral Load Elevation and Associated Liver Disease/Cancer dataset from Taiwan to investigate the causal effect of hepatitis C virus infection on mortality.

Background: Vaccination safety is critical for individual and public health. Many existing methods have been used to conduct safety studies with the VAERS (Vaccine Adverse Event Reporting System) database. However, these methods frequently identify many adverse event (AE) signals and they are often hard to interpret in a biological context. The AE ontology introduces biologically meaningful structures to the VAERS database by connecting similar AEs, which provides meaningful interpretation for the underlying safety issues. In this paper, we develop rigorous statistical methods to identify interesting AE groups by performing AE enrichment analysis. Results: We extend existing gene enrichment tests to perform AE enrichment analysis. Unlike the continuous gene expression data, AE data are counts. Therefore, AE data has many zeros and ties. We propose two enrichment tests, AEFisher and AEKS. AEFisher is a modified Fisher's exact test based on pre-selected significant AEs, while AEKS is based on a modified Kolmogorov-Smirnov statistic. Both tests incorporate the special features of the AE data. The proposed methods were evaluated using simulation studies and were further illustrated on two studies using VAERS data. Conclusion: By appropriately addressing the issues of ties and excessive zeros in AE count data, our enrichment tests performed well as demonstrated by simulation studies and analyses of VAERS data. The proposed methods were implemented in R package AEenrich and can be installed from the Comprehensive R Archive Network, CRAN, and source code are available at https://github.com/umich-biostatistics/AEenrich.

The nested case-control (NCC) design has been widely adopted as a cost-effective sampling design for biomarker research. Under the NCC design, markers are only measured for the NCC subcohort consisting of all cases and a fraction of the controls selected randomly from the matched risk sets of the cases. Robust methods for evaluating prediction performance of risk models have been derived under the inverse probability weighting framework. The probabilities of samples being included in the NCC cohort can be calculated based on the study design ``a previous study'' or estimated non-parametrically ``a previous study''. Neither strategy works well due to model mis-specification and the curse of dimensionality in practical settings where the sampling does not entirely follow the study design or depends on many factors. In this paper, we propose an alternative strategy to estimate the sampling probabilities based on a varying coefficient model, which attains a balance between robustness and the curse of dimensionality. The complex correlation structure induced by repeated finite risk set sampling makes the standard resampling procedure for variance estimation fail. We propose a perturbation resampling procedure that provides valid interval estimation for the proposed estimators. Simulation studies show that the proposed method performs well in finite samples. We apply the proposed method to the Nurses' Health Study II to develop and evaluate prediction models using clinical biomarkers for cardiovascular risk.

In medical research, the Brier score (BS) and the area under the receiver operating characteristic (ROC) curves (AUC) are two common metrics used to evaluate prediction models of a binary outcome, such as using biomarkers to predict the risk of developing a disease in the future. The assessment of an existing prediction models using data with missing covariate values is challenging. In this article, we propose inverse probability weighted (IPW) and augmented inverse probability weighted (AIPW) estimates of AUC and BS to handle the missing data. An alternative approach uses multiple imputation (MI), which requires a model for the distribution of the missing variable. We evaluated the performance of IPW and AIPW in comparison with MI in simulation studies under missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR) scenarios. When there are missing observations in the data, MI and IPW can be used to obtain unbiased estimates of BS and AUC if the imputation model for the missing variable or the model for the missingness is correctly specified. MI is more efficient than IPW. Our simulation results suggest that AIPW can be more efficient than IPW, and also achieves double robustness from miss-specification of either the missingness model or the imputation model. The outcome variable should be included in the model for the missing variable under all scenarios, while it only needs to be included in missingness model if the missingness depends on the outcome. We illustrate these methods using an example from prostate cancer.

In cancer studies, it is important to understand disease heterogeneity among patients so that precision medicine can particularly target high-risk patients at the right time. Many feature variables such as demographic variables and biomarkers, combined with a patient’s survival outcome, can be used to infer such latent heterogeneity. In this work, we propose a mixture model to model each patient’s latent survival pattern, where the mixing probabilities for latent groups are modelled through a multinomial distribution. The Bayesian information criterion (BIC) is used for selecting the number of latent groups. Furthermore, we incorporate variable selection with the adaptive lasso into inference so that only a few feature variables will be selected to characterize the latent heterogeneity. We show that our adaptive lasso estimator has oracle properties when the number of parameters diverges with the sample size. The finite sample performance is evaluated by the simulation study, and the proposed method is illustrated by two datasets.

In clinical or epidemiological follow-up studies, methods based on time scale indicators such as the restricted mean survival time (RMST) have been developed to some extent. Compared with traditional hazard rate indicator system methods, the RMST is easier to interpret and does not require the proportional hazard assumption. To date, regression models based on the RMST are indirect or direct models of the RMST and baseline covariates. However, time-dependent covariates are becoming increasingly common in follow-up studies. Based on the inverse probability of censoring weighting (IPCW) method, we developed a regression model of the RMST and time-dependent covariates. Through Monte Carlo simulation, we verified the estimation performance of the regression parameters of the proposed model. Compared with the time-dependent Cox model and the fixed (baseline) covariate RMST model, the time-dependent RMST model has a better prediction ability. Finally, an example of heart transplantation was used to verify the above conclusions.

The probability of agreement has been used as an effective strategy for quantifying the similarity between the reliability of two populations. By contrast to hypothesis testing approaches based on P-values, the probability of agreement provides a more realistic assessment of similarity by emphasizing practically important differences. In this article, we propose the use of the probability of agreement to evaluate the similarity of two Kaplan-Meier curves, which estimate the survival functions in two populations. This article extends the probability of agreement paradigm to right censored data and explores three different methods of quantifying uncertainty in the probability of agreement estimate. The first approach provides a convenient assessment based on large-sample normal-theory (LSNT), while the other two approaches are nonparametric alternatives based on ordinary and fractional random-weight bootstrap (FRWB) techniques. All methods are illustrated with examples for which comparing the survival curves of related populations is of interest and the efficacy of the methods are also evaluated through simulation studies. Based on these simulations we recommend point estimation using the proposed LSNT calculation and confidence interval estimation via the FRWB approach. We also provide a Shiny app that facilitates an automated implementation of the methodology.

The well-known agreement interval by Bland and Altman is extensively applied in method comparison studies. Two clinical measurement methods are considered interchangeable if their differences are not clinically significant. The agreement interval is commonly applied to assess the spread of the differences. However, this interval is approximate (too narrow) and several authors propose calculating a confidence interval around each bound. This article demonstrates that this approach is misleading, awkward, and confusing. On the other hand, tolerance intervals are exact and can include a confidence level if needed. Tolerance intervals are also easier to calculate and to interpret. Real data sets are used to illustrate the tolerance intervals with the R package BivRegBLS under normal or log-normal assumptions. Furthermore, it is also explained how to assess the coverage probabilities of the tolerance intervals with simulations.

In a longitudinal study, measures of key variables might be incomplete or partially recorded due to drop-out, loss to follow-up, or early termination of the study occurring before the advent of the event of interest. In this paper, we focus primarily on the implementation of a regression model with a randomly censored predictor. We examine, particularly, the use of inverse probability weighting methods in a generalized linear model (GLM), when the predictor of interest is right-censored, to adjust for censoring. To improve the performance of the complete-case analysis and prevent selection bias, we consider three different weighting schemes: inverse censoring probability weights, Kaplan-Meier weights, and Cox proportional hazards weights. We use Monte Carlo simulation studies to evaluate and compare the empirical properties of different weighting estimation methods. Finally, we apply these methods to the Framingham Heart Study data as an illustrative example to estimate the relationship between age of onset of a clinically diagnosed cardiovascular event and low-density lipoprotein among cigarette smokers.

Dynamic treatment regimes operationalize precision medicine as a sequence of decision rules, one per stage of clinical intervention, that map up-to-date patient information to a recommended intervention. An optimal treatment regime maximizes the mean utility when applied to the population of interest. Methods for estimating an optimal treatment regime assume the data to be fully observed, which rarely occurs in practice. A common approach is to first use multiple imputation and then pool the estimators across imputed datasets. However, this approach requires estimating the joint distribution of patient trajectories, which can be high-dimensional, especially when there are multiple stages of intervention. We examine the application of inverse probability weighted estimating equations as an alternative to multiple imputation in the context of monotonic missingness. This approach applies to a broad class of estimators of an optimal treatment regime including both Q-learning and a generalization of outcome weighted learning. We establish consistency under mild regularity conditions and demonstrate its advantages in finite samples using a series of simulation experiments and an application to a schizophrenia study.

Longitudinal biomarker data are often collected in studies, providing important information regarding the probability of an outcome of interest occurring at a future time. With many new and evolving technologies for biomarker discovery, the number of biomarker measurements available for analysis of disease progression has increased dramatically. A large amount of data provides a more complete picture of a patient's disease progression, potentially allowing us to make more accurate and reliable predictions, but the magnitude of available data introduces challenges to most statistical analysts. Existing approaches suffer immensely from the curse of dimensionality. In this article, we propose methods for making dynamic risk predictions using repeatedly measured biomarkers of a large dimension, including cases when the number of biomarkers is close to the sample size. The proposed methods are computationally simple, yet sufficiently flexible to capture complex relationships between longitudinal biomarkers and potentially censored events times. The proposed approaches are evaluated by extensive simulation studies and are further illustrated by an application to a data set from the Nephrotic Syndrome Study Network.

Many diseases such as cancer and heart diseases are heterogeneous and it is of great interest to study the disease risk specific to the subtypes in relation to genetic and environmental risk factors. However, due to logistic and cost reasons, the subtype information for the disease is missing for some subjects. In this paper, we investigate methods for multinomial logistic regression with missing outcome data, including a bootstrap hot deck multiple imputation (BHMI), simple inverse probability weighted (SIPW), augmented inverse probability weighted (AIPW), and expected estimating equation (EEE) estimators. These methods are important approaches for missing data regression. The BHMI modifies the standard hot deck multiple imputation method such that it can provide valid confidence interval estimation. Under the situation when the covariates are discrete, the SIPW, AIPW and EEE estimators are numerically identical. When the covariates are continuous, nonparametric smoothers can be applied to estimate the selection probabilities and the estimating scores. These methods perform similarly. Extensive simulations show that all of these methods yield unbiased estimators while the complete-case analysis can be biased if the missingness depends on the observed data. Our simulations also demonstrate that these methods can gain substantial efficiency compared to the complete-case analysis. The methods are applied to a colorectal cancer study in which cancer subtype data are missing among some study individuals.

In personalized medicine, it is often desired to determine if all patients or only a subset of them benefit from a treatment. We consider estimation in two-stage adaptive designs that in stage 1 recruit patients from the full population. In stage 2, patient recruitment is restricted to the part of the population, which, based on stage 1 data, benefits from the experimental treatment. Existing estimators, which adjust for using stage 1 data for selecting the part of the population from which stage 2 patients are recruited, as well as for the confirmatory analysis after stage 2, do not consider time to event patient outcomes. In this work, for time to event data, we have derived a new asymptotically unbiased estimator for the log hazard ratio and a new interval estimator with good coverage probabilities and probabilities that the upper bounds are below the true values. The estimators are appropriate for several selection rules that are based on a single or multiple biomarkers, which can be categorical or continuous.

peer-reviewed The full text of this article will not be available in ULIR until the embargo expires on the 24/04/2021 We develop flexible multiparameter regression (MPR) survival models for interval‐censored survival data arising in longitudinal prospective studies and longitudinal randomised controlled clinical trials. A multiparameter Weibull regression survival model, which is wholly parametric, and has nonproportional hazards, is the main focus of the article. We describe the basic model, develop the interval‐censored likelihood, and extend the model to include gamma frailty and a dispersion model. We evaluate the models by means of a simulation study and a detailed reanalysis of data from the Signal Tandmobiel study. The results demonstrate that the MPR model with frailty is computationally efficient and provides an excellent fit to the data.

A Bayesian phase I-II dose-finding design is presented for a clinical trial with four coprimary outcomes that reflect the actual clinical observation process. During a prespecified fixed follow-up period, the times to disease progression, toxicity, and death are monitored continuously, and an ordinal disease status variable, including progressive disease (PD) as one level, is evaluated repeatedly by scheduled imaging. We assume a proportional hazards model with piecewise constant baseline hazard for each continuous variable and a longitudinal multinomial probit model for the ordinal disease status process and include multivariate patient frailties to induce association among the outcomes. A finite partition of the nonfatal outcome combinations during the follow-up period is constructed, and the utility of each set in the partition is elicited. Posterior mean utility is used to optimize each patient's dose, subject to a safety rule excluding doses with an unacceptably high rate of PD, severe toxicity, or death. A simulation study shows that, compared with the proposed design, a simpler design based on commonly used efficacy and toxicity outcomes obtained by combining the four variables described above performs poorly and has substantially smaller probabilities of correctly choosing truly optimal doses and excluding truly unsafe doses.

When treatment effect modifiers influence the decision to participate in a randomized trial, the average treatment effect in the population represented by the randomized individuals will differ from the effect in other populations. In this tutorial, we consider methods for extending causal inferences about time-fixed treatments from a trial to a new target population of nonparticipants, using data from a completed randomized trial and baseline covariate data from a sample from the target population. We examine methods based on modeling the expectation of the outcome, the probability of participation, or both (doubly robust). We compare the methods in a simulation study and show how they can be implemented in software. We apply the methods to a randomized trial nested within a cohort of trial-eligible patients to compare coronary artery surgery plus medical therapy versus medical therapy alone for patients with chronic coronary artery disease. We conclude by discussing issues that arise when using the methods in applied analyses.

In the context of survival analysis, calibration refers to the agreement between predicted probabilities and observed event rates or frequencies of the outcome within a given duration of time. We aimed to describe and evaluate methods for graphically assessing the calibration of survival models. We focus on hazard regression models and restricted cubic splines in conjunction with a Cox proportional hazards model. We also describe modifications of the Integrated Calibration Index, of E50 and of E90. In this context, this is the average (respectively, median or 90th percentile) absolute difference between predicted survival probabilities and smoothed survival frequencies. We conducted a series of Monte Carlo simulations to evaluate the performance of these calibration measures when the underlying model has been correctly specified and under different types of model mis‐specification. We illustrate the utility of calibration curves and the three calibration metrics by using them to compare the calibration of a Cox proportional hazards regression model with that of a random survival forest for predicting mortality in patients hospitalized with heart failure. Under a correctly specified regression model, differences between the two methods for constructing calibration curves were minimal, although the performance of the method based on restricted cubic splines tended to be slightly better. In contrast, under a mis‐specified model, the smoothed calibration curved constructed using hazard regression tended to be closer to the true calibration curve. The use of calibration curves and of these numeric calibration metrics permits for a comprehensive comparison of the calibration of competing survival models.

Cystic fibrosis (CF) is a progressive, genetic disease characterized by frequent, prolonged drops in lung function. Accurately predicting rapid underlying lung-function decline is essential for clinical decision support and timely intervention. Determining whether an individual is experiencing a period of rapid decline is complicated due to its heterogeneous timing and extent, and error component of the measured lung function. We construct individualized predictive probabilities for "nowcasting" rapid decline. We assume each patient's true longitudinal lung function, S(t), follows a nonlinear, nonstationary stochastic process, and accommodate between-patient heterogeneity through random effects. Corresponding lung-function decline at time t is defined as the rate of change, S'(t). We predict S'(t) conditional on observed covariate and measurement history by modeling a measured lung function as a noisy version of S(t). The method is applied to data on 30 879 US CF Registry patients. Results are contrasted with a currently employed decision rule using single-center data on 212 individuals. Rapid decline is identified earlier using predictive probabilities than the center's currently employed decision rule (mean difference: 0.65 years; 95% confidence interval (CI): 0.41, 0.89). We constructed a bootstrapping algorithm to obtain CIs for predictive probabilities. We illustrate real-time implementation with R Shiny. Predictive accuracy is investigated using empirical simulations, which suggest this approach more accurately detects peak decline, compared with a uniform threshold of rapid decline. Median area under the ROC curve estimates (Q1-Q3) were 0.817 (0.814-0.822) and 0.745 (0.741-0.747), respectively, implying reasonable accuracy for both. This article demonstrates how individualized rate of change estimates can be coupled with probabilistic predictive inference and implementation for a useful medical-monitoring approach.

Sequential analysis is used in clinical trials and postmarket drug safety surveillance to prospectively monitor efficacy and safety to quickly detect benefits and problems, while taking the multiple testing of repeated analyses into account. When there are multiple outcomes, each one may be given a weight corresponding to its severity. This paper introduces an exact sequential analysis procedure for multiple weighted binomial end points; the analysis incorporates a drug’s combined benefit and safety profile. It works with a variety of alpha spending functions for continuous, group, or mixed group-continuous sequential analysis. The binomial probabilities may vary over time and do not need to be known a priori. The new method was implemented in the free R Sequential package for both one- and two-tailed sequential analysis. An example is given examining myocardial infarction and major bleeding events in patients who initiated non-steroidal antiinflammatory drugs.

A Bayesian phase I-II design is presented that optimizes the dose of a new agent within predefined prognostic subgroups. The design is motivated by a trial to evaluate targeted agents for treating metastatic clear cell renal carcinoma, where a prognostic risk score defined by clinical variables and biomarkers is well established. Two clinical outcomes are used for dose-finding, time-to-toxicity during a prespecified follow-up period, and efficacy characterized by ordinal disease status evaluated at the end of follow-up. A joint probability model is constructed for these outcomes as functions of dose and subgroup. The model performs adaptive clustering of adjacent subgroups having similar dose-outcome distributions to facilitate borrowing information across subgroups. To quantify toxicity-efficacy risk-benefit trade-offs that may differ between subgroups, the objective function is based on outcome utilities elicited separately for each subgroup. In the context of the renal cancer trial, a design is constructed and a simulation study is presented to evaluate the design's reliability, safety, and robustness, and to compare it to designs that either ignore subgroups or run a separate trial within each subgroup.

The population-attributable fraction (PAF) quantifies the public health impact of a harmful exposure. Despite being a measure of significant importance an estimand accommodating complicated time-to-event data is not clearly defined. We discuss current estimands of the PAF used to quantify the public health impact of an internal time-dependent exposure for data subject to competing outcomes. To overcome some limitations, we proposed a novel estimand which is based on dynamic prediction by landmarking. In a profound simulation study, we discuss interpretation and performance of the various estimands and their estimators. The methods are applied to a large French database to estimate the health impact of ventilator-associated pneumonia for patients in intensive care., Comment: A revision has been submitted

In competing risks setting, we account for death according to a specific cause and the quantities of interest are usually the cause-specific hazards (CSHs) and the cause-specific cumulative probabilities. A cause-specific cumulative probability can be obtained with a combination of the CSHs or via the subdistribution hazard. Here, we modeled the CSH with flexible hazard-based regression models using B-splines for the baseline hazard and time-dependent (TD) effects. We derived the variance of the cause-specific cumulative probabilities at the population level using the multivariate delta method and showed how we could easily quantify the impact of a covariate on the cumulative probability scale using covariate-adjusted cause-specific cumulative probabilities and their difference. We conducted a simulation study to evaluate the performance of this approach in its ability to estimate the cumulative probabilities using different functions for the cause-specific log baseline hazard and with or without a TD effect. In the scenario with TD effect, we tested both well-specified and misspecified models. We showed that the flexible regression models perform nearly as well as the nonparametric method, if we allow enough flexibility for the baseline hazards. Moreover, neglecting the TD effect hardly affects the cumulative probabilities estimates of the whole population but impacts them in the various subgroups. We illustrated our approach using data from people diagnosed with monoclonal gammopathy of undetermined significance and provided the R-code to derive those quantities, as an extension of the R-package mexhaz.

Many health issues require adherence to recommended daily activities, such as taking medication to manage a chronic condition, walking a certain distance to promote weight loss, or measuring weights to assess fluid balance in heart failure. The cost of nonadherence can be high, with respect to both individual health outcomes and the healthcare system. Incentivizing adherence to daily activities can promote better health in patients and populations and potentially provide long-term cost savings. Multiple incentive structures are possible. We focus here on a daily lottery incentive in which payment occurs when both the participant's lottery number matches the number drawn and the participant adheres to the targeted daily behavior. Our objective is to model the lottery's effect on participants' probability to complete the targeted task, particularly over the short term. We combine two procedures for analyzing such binary time series: a parameter-driven regression model with an autocorrelated latent process and a comparative interrupted time series. We use the output of the regression model as the control generator for the comparative time series in order to create a quasi-experimental design.

Assessing the calibration of methods for estimating the probability of the occurrence of a binary outcome is an important aspect of validating the performance of risk‐prediction algorithms. Calibration commonly refers to the agreement between predicted and observed probabilities of the outcome. Graphical methods are an attractive approach to assess calibration, in which observed and predicted probabilities are compared using loess‐based smoothing functions. We describe the Integrated Calibration Index (ICI) that is motivated by Harrell's Emax index, which is the maximum absolute difference between a smooth calibration curve and the diagonal line of perfect calibration. The ICI can be interpreted as weighted difference between observed and predicted probabilities, in which observations are weighted by the empirical density function of the predicted probabilities. As such, the ICI is a measure of calibration that explicitly incorporates the distribution of predicted probabilities. We also discuss two related measures of calibration, E50 and E90, which represent the median and 90th percentile of the absolute difference between observed and predicted probabilities. We illustrate the utility of the ICI, E50, and E90 by using them to compare the calibration of logistic regression with that of random forests and boosted regression trees for predicting mortality in patients hospitalized with a heart attack. The use of these numeric metrics permitted for a greater differentiation in calibration than was permissible by visual inspection of graphical calibration curves.

We design two‐stage confirmatory clinical trials that use adaptation to find the subgroup of patients who will benefit from a new treatment, testing for a treatment effect in each of two disjoint subgroups. Our proposal allows aspects of the trial, such as recruitment probabilities of each group, to be altered at an interim analysis. We use the conditional error rate approach to implement these adaptations with protection of overall error rates. Applying a Bayesian decision‐theoretic framework, we optimize design parameters by maximizing a utility function that takes the population prevalence of the subgroups into account. We show results for traditional trials with familywise error rate control (using a closed testing procedure) as well as for umbrella trials in which only the per‐comparison type 1 error rate is controlled. We present numerical examples to illustrate the optimization process and the effectiveness of the proposed designs.

Modeling recurrent event data with multiple event types has drawn interest in recent biomedical studies due to its flexibility for understanding different risk factors for multiple recurrent event processes. However, in such data type, missing event type appears frequently because of various reasons such as recording ignorance or resource limitation. In this study, we aim to propose an inverse probability weighted estimation that is commonly used in the missing data literature to correct possibly biased estimation by a complete-case analysis. This approach is not limited to a specific form of the recurrent event model. We derive the large sample theory in a general form. We demonstrate that our approach can be applied to either multiplicative or additive rates model with practical sample size via comprehensive simulations. Nonmucoid and mucoid Pseudomonas aeruginosa infections of 14 888 patients in 2016 Cystic Fibrosis Foundation Patient Registry data are analyzed to show that, without including 12% events with missing event type in the analysis, several factors may be misidentified as risk factors for the nonmucoid type of infections.

Survival data with competing or semi-competing risks are common in observational studies. As an alternative to cause-specific and subdistribution hazard ratios, the between-group difference in cause-specific restricted mean times lost (RMTL) gives the mean difference in life expectancy lost to a specific cause of death or in disease-free time lost, in the case of a nonfatal outcome, over a pre-specified period. To adjust for covariates, we introduce an inverse probability weighted estimator and its variance for the marginal difference in RMTL. We also introduce an inverse probability of censoring weighted regression model for the RMTL. In simulation studies, we examined the finite sample performance of the proposed methods under proportional and nonproportional subdistribution hazards scenarios. We illustrated both methods with competing risks data from the Framingham Heart Study. We estimated sex differences in atrial fibrillation (AF)-free times lost over 40 years. We also estimated sex differences in mean lifetime lost to cardiovascular disease (CVD) and non-CVD death over 10 years among individuals with AF.

In lifetime data, like cancer studies, there may be long term survivors, which lead to heavy censoring at the end of the follow-up period. Since a standard survival model is not appropriate to handle these data, a cure model is needed. In the literature, covariate hypothesis tests for cure models are limited to parametric and semiparametric methods. We fill this important gap by proposing a nonparametric covariate hypothesis test for the probability of cure in mixture cure models. A bootstrap method is proposed to approximate the null distribution of the test statistic. The procedure can be applied to any type of covariate, and could be extended to the multivariate setting. Its efficiency is evaluated in a Monte Carlo simulation study. Finally, the method is applied to a colorectal cancer dataset. ispartof: STATISTICS IN MEDICINE vol:39 issue:17 pages:2291-2307 ispartof: location:England status: published

Biomarkers are playing an increasingly important role in disease screening, early detection, and risk prediction. The two-phase case-control sampling study design is widely used for the evaluation of candidate biomarkers. The sampling probabilities for cases and controls in the second phase can often depend on other covariates (sampling strata). This biased sampling can lead to invalid inference on a biomarker's classification accuracy if not properly accounted for. In this paper, we adopt the idea of inverse probability weighting and develop inverse probability weighting-based estimators for various measures of a biomarker's classification performance, including the points on the receiver operating characteristics (ROCs) curve, the area under the ROC curve (area under the curve), and the partial area under the curve. In particular, we consider classification accuracy estimators using sampling weights estimated conditionally on sampling strata and further improve their efficiency through the use of estimated weights that additionally take into account the auxiliary variables available from the phase-one cohort. We develop asymptotic properties of the proposed estimators and provide analytical variance for making inference. Extensive simulation studies demonstrate excellent performance of the proposed weighted estimators, while the traditional empirical estimator can be severely biased. We also investigate the advantages in efficiency gain for estimating various classification accuracy estimators through the use of auxiliary variables in addition to sampling strata and apply the proposed method to examples from a renal artery stenosis study and a prostate cancer study.

In the presence of time-dependent confounding there are several methods available\ud to estimate treatment effects. With correctly specified models and appropriate structural\ud assumptions any of these methods could provide consistent effect estimates,\ud but with real-world data all models will be misspecified and it is difficult to know if\ud assumptions are violated.\ud In this paper, we investigate five methods: inverse probability weighting of marginal\ud structural models, history-adjusted marginal structural models, sequential conditional\ud mean models, g-computation formula, and g-estimation of structural nested\ud models. This work is motivated by an investigation of the effects of treatments in cystic\ud fibrosis using the UK Cystic Fibrosis Registry data focussing on two outcomes:\ud lung function (continuous outcome) and annual number of days receiving intravenous\ud antibiotics (count outcome). We identified five features of this data that may affect\ud the performance of the methods: misspecification of the causal null, long-term treatment\ud effects, effect modification by time-varying covariates, misspecification of the\ud direction of causal pathways, and censoring.\ud In simulation studies, under ideal settings all five methods provide consistent estimates\ud of the treatment effect with little difference between methods. However, all\ud methods performed poorly under some settings, highlighting the importance of using\ud appropriate methods based on the data available. Furthermore, with the count outcome,\ud the issue of non-collapsibility makes comparison between methods delivering\ud marginal and conditional effects difficult. In many situations, we would recommend\ud using more than one of the available methods for analysis, as if the effect estimates\ud are very different this would indicate potential issues with the analyses.

There is growing interest in developing clinical prediction models (CPMs) to aid local healthcare decision-making. Frequently, these CPMs are developed in isolation across different populations, with repetitive de novo derivation a common modelling strategy. However, this fails to utilise all available information and does not respond to changes in health processes through time and space. Alternatively, model updating techniques have previously been proposed that adjust an existing CPM to suit the new population, but these techniques are restricted to a single model. Therefore, we aimed to develop a generalised method for updating and aggregating multiple CPMs. The proposed "hybrid method" re-calibrates multiple CPMs using stacked regression while concurrently revising specific covariates using individual participant data (IPD) under a penalised likelihood. The performance of the hybrid method was compared with existing methods in a clinical example of mortality risk prediction after transcatheter aortic valve implantation, and in 2 simulation studies. The simulation studies explored the effect of sample size and between-population-heterogeneity on the method, with each representing a situation of having multiple distinct CPMs and 1 set of IPD. When the sample size of the IPD was small, stacked regression and the hybrid method had comparable but highest performance across modelling methods. Conversely, in large IPD samples, development of a new model and the hybrid method gave the highest performance. Hence, the proposed strategy can inform the choice between utilising existing CPMs or developing a model de novo, thereby incorporating IPD, existing research, and prior (clinical) knowledge into the modelling strategy.

In survival analysis, a competing risk is an event whose occurrence precludes the occurrence of the primary event of interest. Outcomes in medical research are frequently subject to competing risks. In survival analysis, there are 2 key questions that can be addressed using competing risk regression models: first, which covariates affect the rate at which events occur, and second, which covariates affect the probability of an event occurring over time. The cause-specific hazard model estimates the effect of covariates on the rate at which events occur in subjects who are currently event-free. Subdistribution hazard ratios obtained from the Fine-Gray model describe the relative effect of covariates on the subdistribution hazard function. Hence, the covariates in this model can also be interpreted as having an effect on the cumulative incidence function or on the probability of events occurring over time. We conducted a review of the use and interpretation of the Fine-Gray subdistribution hazard model in articles published in the medical literature in 2015. We found that many authors provided an unclear or incorrect interpretation of the regression coefficients associated with this model. An incorrect and inconsistent interpretation of regression coefficients may lead to confusion when comparing results across different studies. Furthermore, an incorrect interpretation of estimated regression coefficients can result in an incorrect understanding about the magnitude of the association between exposure and the incidence of the outcome. The objective of this article is to clarify how these regression coefficients should be reported and to propose suggestions for interpreting these coefficients.

For a patient who has survived a surgery, there could be several levels of recovery. Thus, it is reasonable to consider more than two outcomes when monitoring surgical outcome quality. The risk-adjusted cumulative sum (CUSUM) chart based on multiresponses has been developed for monitoring a surgical process with three or more outcomes. However, there is a significant effect of varying risk distributions on the in-control performance of the chart when constant control limits are applied. To overcome this disadvantage, we apply the dynamic probability control limits to the risk-adjusted CUSUM charts for multiresponses. The simulation results demonstrate that the in-control performance of the charts with dynamic probability control limits can be controlled for different patient populations because these limits are determined for each specific sequence of patients. Thus, the use of dynamic probability control limits for risk-adjusted CUSUM charts based on multiresponses allows each chart to be designed for the corresponding patient sequence of a surgeon or a hospital and therefore does not require estimating or monitoring the patients' risk distribution. Copyright © 2017 John Wiley & Sons, Ltd.

In meta‐analysis of odds ratios (ORs), heterogeneity between the studies is usually modelled via the additive random effects model (REM). An alternative, multiplicative REM for ORs uses overdispersion. The multiplicative factor in this overdispersion model (ODM) can be interpreted as an intra‐class correlation (ICC) parameter. This model naturally arises when the probabilities of an event in one or both arms of a comparative study are themselves beta‐distributed, resulting in beta‐binomial distributions. We propose two new estimators of the ICC for meta‐analysis in this setting. One is based on the inverted Breslow‐Day test, and the other on the improved gamma approximation by Kulinskaya and Dollinger (2015, p. 26) to the distribution of Cochran's Q. The performance of these and several other estimators of ICC on bias and coverage is studied by simulation. Additionally, the Mantel‐Haenszel approach to estimation of ORs is extended to the beta‐binomial model, and we study performance of various ICC estimators when used in the Mantel‐Haenszel or the inverse‐variance method to combine ORs in meta‐analysis. The results of the simulations show that the improved gamma‐based estimator of ICC is superior for small sample sizes, and the Breslow‐Day‐based estimator is the best for n⩾100. The Mantel‐Haenszel‐based estimator of OR is very biased and is not recommended. The inverse‐variance approach is also somewhat biased for ORs≠1, but this bias is not very large in practical settings. Developed methods and R programs, provided in the Web Appendix, make the beta‐binomial model a feasible alternative to the standard REM for meta‐analysis of ORs. © 2017 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

This study proposes a time-varying effect model for examining group differences in trajectories of zero-inflated count outcomes. The motivating example demonstrates that this zero-inflated Poisson model allows investigators to study group differences in different aspects of substance use (e.g., the probability of abstinence and the quantity of alcohol use) simultaneously. The simulation study shows that the accuracy of estimation of trajectory functions improves as the sample size increases; the accuracy under equal group sizes is only higher when the sample size is small (100). In terms of the performance of the hypothesis testing, the type I error rates are close to their corresponding significance levels under all settings. Furthermore, the power increases as the alternative hypothesis deviates more from the null hypothesis, and the rate of this increasing trend is higher when the sample size is larger. Moreover, the hypothesis test for the group difference in the zero component tends to be less powerful than the test for the group difference in the Poisson component. Copyright © 2016 John Wiley & Sons, Ltd.

Deep learning is a class of machine learning algorithms that are popular for building risk prediction models. When observations are censored, the outcomes are only partially observed and standard deep learning algorithms cannot be directly applied. We develop a new class of deep learning algorithms for outcomes that are potentially censored. To account for censoring, the unobservable loss function used in the absence of censoring is replaced by a censoring unbiased transformation. The resulting class of algorithms can be used to estimate both survival probabilities and restricted mean survival. We show how the deep learning algorithms can be implemented by adapting software for uncensored data by using a form of response transformation. We provide comparisons of the proposed deep learning algorithms to existing risk prediction algorithms for predicting survival probabilities and restricted mean survival through both simulated datasets and analysis of data from breast cancer patients.

The fundamental difficulty of establishing causal relationships between an exposure and an outcome in observational data involves disentangling causality from confounding factors. This problem underlies much of neighborhoods research, which abounds with studies that consider associations between neighborhood characteristics and health outcomes in longitudinal data. Such analyses are confounded by selection issues; individuals with above average health outcomes (or associated characteristics) may self-select into advantaged neighborhoods. Techniques commonly used to assess causal inferences in observational longitudinal data, such as inverse probability of treatment weighting (IPTW), may be inappropriate in neighborhoods data due to unique characteristics of such data. We advance the IPTW toolkit by introducing a procedure based on a multivariate kernel density function which is more appropriate for neighborhoods data. The proposed weighting method is applied in conjunction with a marginal structural model. Our empirical analyses use longitudinal data from the Health and Retirement Study; our exposure of interest is an index of neighborhood socioeconomic status (NSES), and we examine its influence on cognitive function. Our findings illustrate the importance of the choice of method for IPTW-the comparison weighting methods provide poor balance across the set of covariates (which is not the case for our preferred procedure) and yield misleading results when applied in the outcomes models. The utility of the multivariate kernel is also validated via simulation. In addition, our findings emphasize the importance of IPTW-controlling for covariates within a regression without IPTW indicates that NSES affects cognition, whereas IPTW-weighted models fail to show a statistically significant effect.

Clinical prediction models aim to provide estimates of absolute risk for a diagnostic or prognostic endpoint. Such models may be derived from data from various studies in the context of a meta-analysis. We describe and propose approaches for assessing heterogeneity in predictor effects and predictions arising from models based on data from different sources. These methods are illustrated in a case study with patients suffering from traumatic brain injury, where we aim to predict 6-month mortality based on individual patient data using meta-analytic techniques (15 studies, n = 11 022 patients). The insights into various aspects of heterogeneity are important to develop better models and understand problems with the transportability of absolute risk predictions.