Search

Comparisons between randomized trial analyses and observational analyses that attempt to address similar research questions have generated many controversies in epidemiology and the social sciences. There has been little consensus on when such comparisons are reasonable, what their implications are for the validity of observational analyses, or whether trial and observational analyses can be integrated to address effectiveness questions. Here, we consider methods for using observational analyses to complement trial analyses when assessing treatment effectiveness. First, we review the framework for designing observational analyses that emulate target trials and present an evidence map of its recent applications. We then review approaches for estimating the average treatment effect in the target population underlying the emulation: using observational analyses of the emulation data alone; and using transportability analyses to extend inferences from a trial to the target population. We explain how comparing treatment effect estimates from the emulation against those from the trial can provide evidence on whether observational analyses can be trusted to deliver valid estimates of effectiveness – a process we refer to as benchmarking – and, in some cases, allow the joint analysis of the trial and observational data. We illustrate different approaches using a simplified example of a pragmatic trial and its emulation in registry data. We conclude that synthesizing trial and observational data – in transportability, benchmarking, or joint analyses – can leverage their complementary strengths to enhance learning about comparative effectiveness, through a process combining quantitative methods and epidemiological judgements.

In this paper, we consider methods for generating draws of a binary random variable whose expectation conditional on covariates follows a logistic regression model with known covariate coefficients. We examine approximations for finding a “balancing intercept,” that is, a value for the intercept of the logistic model that leads to a desired marginal expectation for the binary random variable. We show that a recently proposed analytical approximation can produce inaccurate results, especially when targeting more extreme marginal expectations or when the linear predictor of the regression model has high variance. We then formulate the balancing intercept as a solution to an integral equation, implement a numerical approximation for solving the equation based on Monte Carlo methods, and show that the approximation works well in practice. Our approach to the basic problem of the balancing intercept provides an example of a broadly applicable strategy for formulating and solving problems that arise in the design of simulation studies used to evaluate or teach epidemiologic methods.

INTRODUCTION. Previous research has predominately relied on person-level or single characteristics of drinking episodes to characterise patterns of drinking that may confer risk. This research often relies on self-report measures. Advancements in wearable alcohol biosensors provide a multi-faceted objective measure of drinking. The current study aimed to characterise drinking episodes using data derived from a wearable alcohol biosensor. METHODS. Participants (n = 45) were adult heavy drinkers who wore the Secure Continuous Remote Alcohol Monitoring (SCRAM) bracelet and reported on their drinking behaviours. Cluster analysis was used to evaluate unique combinations of alcohol episode characteristics. Associations between clusters and self-reported person and event-level factors were also examined in univariable and multivariable models. RESULTS. Results suggested three unique clusters: Cluster 1 (most common, slowest rate of rise to and decline from peak), Cluster 2 (highest peak transdermal alcohol concentration and area under the curve) and Cluster 3 (fastest rate of decline from peak). Univariable analyses distinguished Cluster 1 as having fewer self-reported drinks and fewer episodes that occurred on weekends relative to Cluster 2. The effect for number of drinks remained in multivariable analyses. DISCUSSION AND CONCLUSIONS. This is the first study to characterise drinking patterns at the event-level using objective data. Results suggest that it is possible to distinguish drinking episodes based on several characteristics derived from wearable alcohol biosensors. This examination lays the groundwork for future studies to characterise patterns of drinking and their association with consequences of drinking behaviour.

When treatment effect modifiers influence the decision to participate in a randomized trial, the average treatment effect in the population represented by the randomized individuals will differ from the effect in other populations. In this tutorial, we consider methods for extending causal inferences about time-fixed treatments from a trial to a new target population of nonparticipants, using data from a completed randomized trial and baseline covariate data from a sample from the target population. We examine methods based on modeling the expectation of the outcome, the probability of participation, or both (doubly robust). We compare the methods in a simulation study and show how they can be implemented in software. We apply the methods to a randomized trial nested within a cohort of trial-eligible patients to compare coronary artery surgery plus medical therapy versus medical therapy alone for patients with chronic coronary artery disease. We conclude by discussing issues that arise when using the methods in applied analyses.

Low-value healthcare is costly and inefficient and may adversely affect patient outcomes. Despite increases in low-value service use, little is known about how the receipt of low-value care differs across payers.To evaluate differences in the use of low-value care between patients with commercial versus Medicaid coverage.Retrospective observational analysis of the 2017 Rhode Island All-payer Claims Database, estimating the probability of receiving each of 14 low-value services between commercial and Medicaid enrollees, adjusting for patient sociodemographic and clinical characteristics. Ensemble machine learning minimized the possibility of model misspecification.Medicaid and commercial enrollees aged 18-64 with continuous coverage and an encounter at which they were at risk of receiving a low-value service.Enrollment in Medicaid or Commercial insurance.Use of one of 14 validated measures of low-value care.Among 110,609 patients, Medicaid enrollees were younger, had more comorbidities, and were more likely to be female than commercial enrollees. Medicaid enrollees had higher rates of use for 7 low-value care measures, and those with commercial coverage had higher rates for 5 measures. Across all measures of low-value care, commercial enrollees received more (risk difference [RD] 6.8 percentage points; CI: 6.6 to 7.0) low-value services than their counterparts with Medicaid. Commercial enrollees were also more likely to receive low-value services typically performed in the emergency room (RD 11.4 percentage points; CI: 10.7 to 12.2) and services that were less expensive (RD 15.3 percentage points; CI 14.6 to 16.0).Differences in the provision of low-value care varied across measures, though average use was slightly higher among commercial than Medicaid enrollees. This difference was more pronounced for less expensive services indicating that financial incentives may not be the sole driver of low-value care.

The uncertainty in predictions from deep neural network analysis of medical imaging is challenging to assess but potentially important to include in subsequent decision-making. Using data from diabetic retinopathy detection, we present an empirical evaluation of the role of model calibration in uncertainty-based referral, an approach that prioritizes referral of observations based on the magnitude of a measure of uncertainty. We consider several configurations of network architecture, methods for uncertainty estimation, and training data size. We identify a strong relationship between the effectiveness of uncertainty-based referral and having a well-calibrated model. This is especially relevant as complex deep neural networks tend to have high calibration errors. Finally, we show that post-calibration of the neural network helps uncertainty-based referral with identifying hard-to-classify observations.

Summary Causally interpretable meta-analysis combines information from a collection of randomized controlled trials to estimate treatment effects in a target population in which experimentation may not be possible but from which covariate information can be obtained. In such analyses, a key practical challenge is the presence of systematically missing data when some trials have collected data on one or more baseline covariates, but other trials have not, such that the covariate information is missing for all participants in the latter. In this article, we provide identification results for potential (counterfactual) outcome means and average treatment effects in the target population when covariate data are systematically missing from some of the trials in the meta-analysis. We propose three estimators for the average treatment effect in the target population, examine their asymptotic properties, and show that they have good finite-sample performance in simulation studies. We use the estimators to analyze data from two large lung cancer screening trials and target population data from the National Health and Nutrition Examination Survey (NHANES). To accommodate the complex survey design of the NHANES, we modify the methods to incorporate survey sampling weights and allow for clustering.

We propose methods for estimating the area under the receiver operating characteristic (ROC) curve (AUC) of a prediction model in a target population that differs from the source population that provided the data used for original model development. If covariates that are associated with model performance, as measured by the AUC, have a different distribution in the source and target populations, then AUC estimators that only use data from the source population will not reflect model performance in the target population. Here, we provide identification results for the AUC in the target population when outcome and covariate data are available from the sample of the source population, but only covariate data are available from the sample of the target population. In this setting, we propose three estimators for the AUC in the target population and show that they are consistent and asymptotically normal. We evaluate the finite-sample performance of the estimators using simulations and use them to estimate the AUC in a nationally representative target population from the National Health and Nutrition Examination Survey for a lung cancer risk prediction model developed using source population data from the National Lung Screening Trial.

SummaryPrediction models are often built and evaluated using data from a population that differs from the target population where model-derived predictions are intended to be used in. In this article, we present methods for evaluating model performance in the target population when some observations are right censored. The methods assume that outcome and covariate data are available from a source population used for model development and covariates, but no outcome data, are available from the target population. We evaluate the finite sample performance of the proposed estimators using simulations and apply the methods to transport a prediction model built using data from a lung cancer screening trial to a nationally representative population of participants eligible for lung cancer screening.

Endowing meta-analytic results with a causal interpretation is challenging when there are differences in the distribution of effect modifiers among the populations underlying the included trials and the target population where the results of the meta-analysis will be applied. Recent work on transportability methods has described identifiability conditions under which the collection of randomized trials in a meta-analysis can be used to draw causal inferences about the target population. When the conditions hold, the methods enable estimation of causal quantities such as the average treatment effect and conditional average treatment effect in target populations that differ from the populations underlying the trial samples. The methods also facilitate comparison of treatments not directly compared in a head-to-head trial and assessment of comparative effectiveness within subgroups of the target population. We briefly describe these methods and present a worked example using individual participant data from three HIV prevention trials among adolescents in mental health care. We describe practical challenges in defining the target population, obtaining individual participant data from included trials and a sample of the target population, and addressing systematic missing data across datasets. When fully realized, methods for causally interpretable meta-analysis can provide decision-makers valid estimates of how treatments will work in target populations of substantive interest as well as in subgroups of these populations.

Objective To measure associations between participation in community-based microfinance groups, retention in HIV care, and death among people with HIV (PWH) in low-resource settings. Design and methods We prospectively analyzed data from 3609 patients enrolled in an HIV care program in western Kenya. HIV patients who were eligible and chose to participate in a Group Integrated Savings for Health Empowerment (GISHE) microfinance group were matched 1 : 2 on age, sex, year of enrollment in HIV care, and location of initial HIV clinic visit to patients not participating in GISHE. Follow-up data were abstracted from medical records from January 2018 through February 2020. Logistic regression analysis examined associations between GISHE participation and two outcomes: retention in HIV care (i.e. >1 HIV care visit attended within 6 months prior to the end of follow-up) and death. Socioeconomic factors associated with HIV outcomes were included in adjusted models. Results The study population was majority women (78.3%) with a median age of 37.4 years. Microfinance group participants were more likely to be retained in care relative to HIV patients not participating in a microfinance group [adjusted odds ratio (aOR) = 1.31, 95% confidence interval (CI) 1.01-1.71; P = 0.046]. Participation in group microfinance was associated with a reduced odds of death during the follow-up period (aOR = 0.57, 95% CI 0.28-1.09; P = 0.105). Conclusion Participation in group-based microfinance appears to be associated with better HIV treatment outcomes. A randomized trial is needed to assess whether microfinance groups can improve clinical and socioeconomic outcomes among PWH in similar settings.

Methods for extending – generalizing or transporting – inferences from a randomized trial to a target population involve conditioning on a large set of covariates that is sufficient for rendering the randomized and non-randomized groups exchangeable. Yet, decision-makers are often interested in examining treatment effects in subgroups of the target population defined in terms of only a few discrete covariates. Here, we propose methods for estimating subgroup-specific potential outcome means and average treatment effects in generalizability and transportability analyses, using outcome model-based (g-formula), weighting, and augmented weighting estimators. We consider estimating subgroup-specific average treatment effects in the target population and its non-randomized subset, and provide methods that are appropriate both for nested and non-nested trial designs. As an illustration, we apply the methods to data from the Coronary Artery Surgery Study to compare the effect of surgery plus medical therapy versus medical therapy alone for chronic coronary artery disease in subgroups defined by history of myocardial infarction.

IntroductionIn Kenya, distance to health facilities, inefficient vertical care delivery and limited financial means are barriers to retention in HIV care. Furthermore, the increasing burden of non-communicable diseases (NCDs) among people living with HIV complicates chronic disease treatment and strains traditional care delivery models. Potential strategies for improving HIV/NCD treatment outcomes are differentiated care, community-based care and microfinance (MF).Methods and analysisWe will use a cluster randomised trial to evaluate integrated community-based (ICB) care incorporated into MF groups in medium and high HIV prevalence areas in western Kenya. We will conduct baseline assessments with n=900 HIV positive members of 40 existing MF groups. Group clusters will be randomised to receive either (1) ICB or (2) standard of care (SOC). The ICB intervention will include: (1) clinical care visits during MF group meetings inclusive of medical consultations, NCD management, distribution of antiretroviral therapy (ART) and NCD medications, and point-of-care laboratory testing; (2) peer support for ART adherence and (3) facility referrals as needed. MF groups randomised to SOC will receive regularly scheduled care at a health facility. Findings from the two trial arms will be compared with follow-up data from n=300 matched controls. The primary outcome will be VS at 18 months. Secondary outcomes will be retention in care, absolute mean change in systolic blood pressure and absolute mean change in HbA1c level at 18 months. We will use mediation analysis to evaluate mechanisms through which MF and ICB care impact outcomes and analyse incremental cost-effectiveness of the intervention in terms of cost per HIV suppressed person-time, cost per patient retained in care and cost per disability-adjusted life-year saved.Ethics and disseminationThe Moi University Institutional Research and Ethics Committee approved this study (IREC#0003054). We will share data via the Brown University Digital Repository and disseminate findings via publication.Trial registration numberNCT04417127.

BACKGROUND: HIV molecular epidemiology is increasingly integrated into public health prevention. We conducted cluster typing to enhance characterization of a densely-sampled statewide epidemic towards informing public health. METHODS. We identified HIV clusters, categorized them into types, and evaluated their dynamics between 2004–2019 in Rhode Island (RI). We grouped sequences by diagnosis year, assessed cluster changes between paired phylogenies, t(0) and t(1), representing adjacent years and categorized clusters as stable (cluster in t(0) phylogeny = cluster in t(1) phylogeny) or unstable (cluster in t(0) ≠ cluster in t(1)). Unstable clusters were further categorized as emerging (t(1) phylogeny only) or growing (larger in t(1) phylogeny). We determined proportions of each cluster type, of individuals in each cluster type, and of newly-diagnosed individuals in each cluster type, and assessed trends over time. RESULTS. A total of 1,727 individuals with available HIV-1 subtype B pol sequences were diagnosed in RI by 2019. Over time, stable clusters and individuals in them dominated the epidemic, increasing over time, with reciprocally decreasing unstable clusters and individuals in them. Conversely, proportions of newly-diagnosed individuals in unstable clusters significantly increased. Within unstable clusters, proportions of emerging clusters and of individuals in them declined; while proportions of newly-diagnosed individuals in growing clusters significantly increased over time. CONCLUSIONS. Distinct molecular cluster types were identified in the RI epidemic. Cluster dynamics demonstrated increasing stable and decreasing unstable clusters driven by growing, rather than emerging clusters, suggesting consistent in-state transmission networks. Cluster typing could inform public health beyond conventional approaches and direct interventions.

We considered methods for transporting a prediction model for use in a new target population, both when outcome and covariate data for model development are available from a source population that has a different covariate distribution compared with the target population and when covariate data (but not outcome data) are available from the target population. We discuss how to tailor the prediction model to account for differences in the data distribution between the source population and the target population. We also discuss how to assess the model’s performance (e.g., by estimating the mean squared prediction error) in the target population. We provide identifiability results for measures of model performance in the target population for a potentially misspecified prediction model under a sampling design where the source and the target population samples are obtained separately. We introduce the concept of prediction error modifiers that can be used to reason about tailoring measures of model performance to the target population. We illustrate the methods in simulated data and apply them to transport a prediction model for lung cancer diagnosis from the National Lung Screening Trial to the nationally representative target population of trial-eligible individuals in the National Health and Nutrition Examination Survey.

Introduction Our goal was to determine if features of surgical patients, easily obtained from the medical chart or brief interview, could be used to predict those likely to experience more rapid cognitive decline following surgery. Methods We analyzed data from an observational study of 560 older adults (≥70 years) without dementia undergoing major elective non‐cardiac surgery. Cognitive decline was measured using change in a global composite over 2 to 36 months following surgery. Predictive features were identified as variables readily obtained from chart review or a brief patient assessment. We developed predictive models for cognitive decline (slope) and predicting dichotomized cognitive decline at a clinically determined cut. Results In a hold‐out testing set, the regularized regression predictive model achieved a root mean squared error (RMSE) of 0.146 and a model r‐square (R2) of .31. Prediction of “rapid” decliners as a group achieved an area under the curve (AUC) of .75. Conclusion Some of our models could predict persons with increased risk for accelerated cognitive decline with greater accuracy than relying upon chance, and this result might be useful for stratification of surgical patients for inclusion in future clinical trials.

Methods for extending -- generalizing or transporting -- inferences from a randomized trial to a target population involve conditioning on a large set of covariates that is sufficient for rendering the randomized and non-randomized groups exchangeable. Yet, decision-makers are often interested in examining treatment effects in subgroups of the target population defined in terms of only a few discrete covariates. Here, we propose methods for estimating subgroup-specific potential outcome means and average treatment effects in generalizability and transportability analyses, using outcome model-based (g-formula), weighting, and augmented weighting estimators. We consider estimating subgroup-specific average treatment effects in the target population and its non-randomized subset, and provide methods that are appropriate both for nested and non-nested trial designs. As an illustration, we apply the methods to data from the Coronary Artery Surgery Study to compare the effect of surgery plus medical therapy versus medical therapy alone for chronic coronary artery disease in subgroups defined by history of myocardial infarction.

Public health interventions guided by clustering of HIV-1 molecular sequences may be impacted by choices of analytical approaches. We identified commonly-used clustering analytical approaches, applied them to 1886 HIV-1 Rhode Island sequences from 2004–2018, and compared concordance in identifying molecular HIV-1 clusters within and between approaches. We used strict (topological support ≥ 0.95; distance 0.015 substitutions/site) and relaxed (topological support 0.80–0.95; distance 0.030–0.045 substitutions/site) thresholds to reflect different epidemiological scenarios. We found that clustering differed by method and threshold and depended more on distance than topological support thresholds. Clustering concordance analyses demonstrated some differences across analytical approaches, with RAxML having the highest (91%) mean summary percent concordance when strict thresholds were applied, and three (RAxML-, FastTree regular bootstrap- and IQ-Tree regular bootstrap-based) analytical approaches having the highest (86%) mean summary percent concordance when relaxed thresholds were applied. We conclude that different analytical approaches can yield diverse HIV-1 clustering outcomes and may need to be differentially used in diverse public health scenarios. Recognizing the variability and limitations of commonly-used methods in cluster identification is important for guiding clustering-triggered interventions to disrupt new transmissions and end the HIV epidemic.

BACKGROUND: Our objective was to assess the performance of machine learning methods to predict post-operative delirium using a prospective clinical cohort. METHODS: We analyzed data from an observational cohort study of 560 older adults (≥ 70 years) without dementia undergoing major elective non-cardiac surgery. Post-operative delirium was determined by the Confusion Assessment Method supplemented by a medical chart review (N = 134, 24%). Five machine learning algorithms and a standard stepwise logistic regression model were developed in a training sample (80% of participants) and evaluated in the remaining hold-out testing sample. We evaluated three overlapping feature sets, restricted to variables that are readily available or minimally burdensome to collect in clinical settings, including interview and medical record data. A large feature set included 71 potential predictors. A smaller set of 18 features was selected by an expert panel using a consensus process, and this smaller feature set was considered with and without a measure of pre-operative mental status. RESULTS: The area under the receiver operating characteristic curve (AUC) was higher in the large feature set conditions (range of AUC, 0.62–0.71 across algorithms) versus the selected feature set conditions (AUC range, 0.53–0.57). The restricted feature set with mental status had intermediate AUC values (range, 0.53–0.68). In the full feature set condition, algorithms such as gradient boosting, cross-validated logistic regression, and neural network (AUC = 0.71, 95% CI 0.58–0.83) were comparable with a model developed using traditional stepwise logistic regression (AUC = 0.69, 95% CI 0.57–0.82). Calibration for all models and feature sets was poor. CONCLUSIONS: We developed machine learning prediction models for post-operative delirium that performed better than chance and are comparable with traditional stepwise logistic regression. Delirium proved to be a phenotype that was difficult to predict with appreciable accuracy. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s11606-020-06238-7) contains supplementary material, which is available to authorized users.

Great efforts are devoted to end the HIV epidemic as it continues to have profound public health consequences in the United States and throughout the world, and new interventions and strategies are continuously needed. The use of HIV sequence data to infer transmission networks holds much promise to direct public heath interventions where they are most needed. As these new methods are being implemented, evaluating their benefits is essential. In this paper, we recognize challenges associated with such evaluation, and make the case that overcoming these challenges is key to the use of HIV sequence data in routine public health actions to disrupt HIV transmission networks.

We introduce causal interaction tree (CIT) algorithms for finding subgroups of individuals with heterogeneous treatment effects in observational data. The CIT algorithms are extensions of the classification and regression tree algorithm that use splitting criteria based on subgroup-specific treatment effect estimators appropriate for observational data. We describe inverse probability weighting, g-formula, and doubly robust estimators of subgroup-specific treatment effects, derive their asymptotic properties, and use them to construct splitting criteria for the CIT algorithms. We study the performance of the algorithms in simulations and implement them to analyze data from an observational study that evaluated the effectiveness of right heart catheterization for critically ill patients.

IntroductionHIV continues to have great impact on millions of lives. Novel methods are needed to disrupt HIV transmission networks. In the USA, public health departments routinely conduct contact tracing and partner services and interview newly HIV-diagnosed index cases to obtain information on social networks and guide prevention interventions. Sequence clustering methods able to infer HIV networks have been used to investigate and halt outbreaks. Incorporation of such methods into routine, not only outbreak-driven, contact tracing and partner services holds promise for further disruption of HIV transmissions.Methods and analysisBuilding on a strong academic–public health collaboration in Rhode Island, we designed and have implemented a state-wide prospective study to evaluate an intervention that incorporates real-time HIV molecular clustering information with routine contact tracing and partner services. We present the rationale and study design of our approach to integrate sequence clustering methods into routine public health interventions as well as related important ethical considerations. This prospective study addresses key questions about the benefit of incorporating a clustering analysis triggered intervention into the routine workflow of public health departments, going beyond outbreak-only circumstances. By developing an intervention triggered by, and incorporating information from, viral sequence clustering analysis, and evaluating it with a novel design that avoids randomisation while allowing for methods comparison, we are confident that this study will inform how viral sequence clustering analysis can be routinely integrated into public health to support the ending of the HIV pandemic in the USA and beyond.Ethics and disseminationThe study was approved by both the Lifespan and Rhode Island Department of Health Human Subjects Research Institutional Review Boards and study results will be published in peer-reviewed journals.

This article proposes a novel paradigm for building regression trees and ensemble learning in survival analysis. Generalizations of the classification and regression trees (CART) and random forests (RF) algorithms for general loss functions, and in the latter case more general bootstrap procedures, are both introduced. These results, in combination with an extension of the theory of censoring unbiased transformations (CUTs) applicable to loss functions, underpin the development of two new classes of algorithms for constructing survival trees and survival forests: censoring unbiased regression trees and censoring unbiased regression ensembles. For a certain “doubly robust” CUT of squared error loss, we further show how these new algorithms can be implemented using existing software (e.g., CART, RF). Comparisons of these methods to existing ensemble procedures for predicting survival probabilities are provided in both simulated settings and through applications to four datasets. It is shown that these new methods either improve upon, or remain competitive with, existing implementations of random survival forests, conditional inference forests, and recursively imputed survival trees.

Validated biomarkers that more accurately predict prognosis and/or measure disease burden in patients with high-grade gliomas would help triage which treatment strategies are most promising for evaluation in Phase III multicenter trials. Multicenter trials to evaluate imaging biomarkers in this group face particular challenges; these trials have historically been slow to accrue and have not recently succeeded in validating new imaging biomarkers useful in treatment development. Due to variability in image acquisition protocols, scanner hardware, image analysis, and interpretive schemes, promising results obtained in single centers are poor predictors of success in the multicenter setting. Multicenter preliminary data to support further evaluation of imaging biomarkers is rarely available. The need for more efficient trial designs that bring multicenter data earlier into the process of biomarker development has become increasingly clear. In this presentation, the planning process within ECOG-ACRIN’s Brain Tumor Working Group for a platform multicenter trial called GABLE (Glioblastoma Accelerated Biomarker Learning Environment trial) designed to evaluate biomarkers for distinguishing pseudoprogression from true progression in patients with newly diagnosed GBM is described. In our planning process, it was determined that efficiencies can be gained from evaluating multiple biomarker types in parallel rather than serially; in the context of the proposed trial, not only conventional imaging biomarkers but plasma biomarkers and radiomic biomarkers can be evaluated simultaneously. Patient tolerance limits the feasibility of evaluating multiple non-standard-of-care imaging biomarkers in parallel. For this group of biomarkers, a “fast-switching” serial evaluation strategy using multiple interim analyses was developed to triage out biomarkers unlikely to succeed in identifying patient groups with clinically significant differences in median survival. For biomarker triage, an endpoint of event-free survival (events of either death or NANO progression) was proposed. Simulations were used to evaluate alpha and beta error using this evaluation strategy.

This article considers linear regression with missing covariates and a right censored outcome. We first consider a general two-phase outcome sampling design, where full covariate information is only ascertained for subjects in phase two and sampling occurs under an independent Bernoulli sampling scheme with known subject-specific sampling probabilities that depend on phase one information (e.g., survival time, failure status and covariates). The semiparametric information bound is derived for estimating the regression parameter in this setting. We also introduce a more practical class of augmented estimators that is shown to improve asymptotic efficiency over simple but inefficient inverse probability of sampling weighted estimators. Estimation for known sampling weights and extensions to the case of estimated sampling weights are both considered. The allowance for estimated sampling weights permits covariates to be missing at random according to a monotone but unknown mechanism. The asymptotic properties of the augmented estimators are derived and simulation results demonstrate substantial efficiency improvements over simpler inverse probability of sampling weighted estimators in the indicated settings. With suitable modification, the proposed methodology can also be used to improve augmented estimators previously used for missing covariates in a Cox regression model. Supplementary materials for this article are available online.

RESEARCH OBJECTIVE: In cluster‐randomized trials, participating clusters are not necessarily representative of the target population of trial‐eligible clusters where the experimental treatments might be applied. To the extent that participating and nonparticipating clusters differ in terms of covariates that are modifiers of the treatment effect, the average treatment effect in the trial may not apply to the entire target population. We describe analyses that extend (generalize or transport) causal inferences from cluster‐randomized trials to a target population of clusters, under a general nonparametric model that allows for arbitrary within‐cluster dependence. STUDY DESIGN: We consider study designs where a subset of a cohort of clusters are invited and agree to participate in a randomized trial with cluster‐level treatment assignment (ie, nested trial designs). Treatment and outcome data need only be collected from clusters participating in the trial, but data on baseline covariates must be collected from the entire cohort. We propose doubly robust estimators of potential outcome means in the target population that exploit individual‐level data on covariates and outcomes to improve efficiency and are appropriate for use with machine learning methods. POPULATION STUDIED: We illustrate the methods using a 2 × 2 factorial cluster‐randomized trial of influenza vaccination strategies conducted in 818 nursing homes nested in a cohort of 4,475 trial‐eligible Medicare‐certified nursing homes. To be eligible for the trial, nursing homes had to be free‐standing facilities (ie, not hospital based) within 50 miles of a CDC‐reporting city, have a minimum of 50 long‐stay residents, and at least 80% of residents aged 65 years or older. PRINCIPAL FINDINGS: We applied the methods to estimate the potential outcome means for the four vaccination strategies in the target population of trial‐eligible nursing homes. We estimated the conditional probability of trial participation (nursing home level) and the conditional probability of death from any cause (individual level, separately for each treatment group) using different methods to illustrate that our approach can easily accommodate them. One set of analyses used logistic regression models estimated by maximum likelihood methods. For comparison, we also estimated probabilities using various machine learning methods. Overall, the doubly robust estimator produced similar point estimates for the potential outcome mean under each treatment across all approaches for estimating the conditional probabilities of trial participation and death. Point estimates for generalizability analyses were largely similar to those from the randomized trial, suggesting that the trial‐only estimates were applicable to the broader population of trial‐eligible nursing homes for the outcome of all‐cause death. CONCLUSIONS: Recent work on extending causal inferences from individually randomized trials to a target population can be extended to the cluster trial setting, under a general causal model that allows for arbitrary within‐cluster dependence. IMPLICATIONS FOR POLICY OR PRACTICE: When cluster‐randomized trials are embedded in large health care systems, investigators can use the methods we describe to extend causal inferences from the trials to the broader health care system. PRIMARY FUNDING SOURCE: Patient‐Centered Outcomes Research Institute.

In this paper, we present a method for conducting global sensitivity analysis of randomized trials in which binary outcomes are scheduled to be collected on participants at pre-specified points in time after randomization and these outcomes may be missing in a non-monotone fashion. We introduce a class of missing data assumptions, indexed by sensitivity parameters, that are anchored around the missing not at random assumption introduced by Robins (Statistics in Medicine, 1997). For each assumption in the class, we establish that the joint distribution of the outcomes are identifiable from the distribution of the observed data. Our estimation procedure uses the plug-in principle, where the distribution of the observed data is estimated using random forests. We establish √n asymptotic properties for our estimation procedure. We illustrate our methodology in the context of a randomized trial designed to evaluate a new approach to reducing substance use, assessed by testing urine samples twice weekly, among patients entering outpatient addiction treatment. We evaluate the finite sample properties of our method in a realistic simulation study. Our methods have been implemented in an R package entitled slabm. This article is protected by copyright. All rights reserved.

Publisher's version (útgefin grein), Little is known about self-rated health (SRH) of older people living in more remote and Arctic areas. Iceland is a high-income country with one of the lowest rates of income inequality in the world, which may influence SRH. The research aim was to study factors affecting SRH, in such a population living in Northern Iceland. Stratified random sample according to the place of residency, age and gender was used and data collected via face-to-face interviews. Inclusion criteria included community-dwelling adults ≥65 years of age. Response rate was 57.9% (N = 175), average age 74.2 (sd 6.3) years, range 65–92 years and 57% were men. The average number of diagnosed diseases was 1.5 (sd 1.3) and prescribed medications 3.0 (sd 1.7). SRH ranged from 5 (excellent) to 1 (bad), with an average of 3.26 (sd 1.0) and no difference between the place of residency. Lower SRH was independently explained by depressed mood (OR = 0.88, 95% CI = 0.80–0.96), higher body mass index (OR = 0.93, 95% CI = 0.87–0.99), number of prescribed medications (OR = 0.88, 95% CI = 0.78–1.00) and perception of inadequate income (OR = 0.45, 95% CI = 0.21–0.98). The results highlight the importance of physical and mental health promotion for general health and for ageing in place and significance of economic factors as predictors of SRH., This work was supported by the Háskólinn á Akureyri [R-1803]; Icelandic Regional Development Institute (Byggðastofnun) [102022].

Objectives: National Comprehensive Cancer Network (NCCN) guidelines for stage III colon cancer define low-risk versus high-risk patients based on T (1 to 3 vs. 4) and N (1 vs. 2) status, with some variations in treatment. This study analyzes the impact of tumor deposits (TDs), T and N status, poor differentiation (PD), perineural invasion (PNI), and lymphovascular invasion (LVI) on survival. Materials and Methods: A retrospective analysis (2010-2015) of the National Cancer Database of stage III colon cancer patients treated with both surgery and chemotherapy was conducted. Data was extracted on sex, race, age at diagnosis, Charlson-Deyo Score, histopathologic variables, and survival rates. Statistical analysis used the test of proportions, log-rank test for Kaplan-Meier curves, and Cox proportional hazard models. Results: For the 42,901 patients analyzed, 5-year survival rates were similar for LN+TD− (59.8%) and LN−TD+ (58.2%), but significantly worse for LN+TD+ (41.5%) (P

Deep learning is a class of machine learning algorithms that are popular for building risk prediction models. When observations are censored, the outcomes are only partially observed and standard deep learning algorithms cannot be directly applied. We develop a new class of deep learning algorithms for outcomes that are potentially censored. To account for censoring, the unobservable loss function used in the absence of censoring is replaced by a censoring unbiased transformation. The resulting class of algorithms can be used to estimate both survival probabilities and restricted mean survival. We show how the deep learning algorithms can be implemented by adapting software for uncensored data by using a form of response transformation. We provide comparisons of the proposed deep learning algorithms to existing risk prediction algorithms for predicting survival probabilities and restricted mean survival through both simulated datasets and analysis of data from breast cancer patients.

Estimating a patient's mortality risk is important in making treatment decisions. Survival trees are a useful tool and employ recursive partitioning to separate patients into different risk groups. Existing 'loss based' recursive partitioning procedures that would be used in the absence of censoring have previously been extended to the setting of right censored outcomes using inverse probability censoring weighted estimators of loss functions. In this paper, we propose new 'doubly robust' extensions of these loss estimators motivated by semiparametric efficiency theory for missing data that better utilize available data. Simulations and a data analysis demonstrate strong performance of the doubly robust survival trees compared with previously used methods. Copyright © 2016 John Wiley & Sons, Ltd.

We present methods for causally interpretable meta-analyses that combine information from multiple randomized trials to estimate potential (counterfactual) outcome means and average treatment effects in a target population. We consider identifiability conditions, derive implications of the conditions for the law of the observed data, and obtain identification results for transporting causal inferences from a collection of independent randomized trials to a new target population in which experimental data may not be available. We propose an estimator for the potential (counterfactual) outcome mean in the target population under each treatment studied in the trials. The estimator uses covariate, treatment, and outcome data from the collection of trials, but only covariate data from the target population sample. We show that it is doubly robust, in the sense that it is consistent and asymptotically normal when at least one of the models it relies on is correctly specified. We study the finite sample properties of the estimator in simulation studies and demonstrate its implementation using data from a multi-center randomized trial.

Summary We consider the problem of designing a confirmatory randomized trial for comparing two treatments versus a common control in two disjoint subpopulations. The subpopulations could be defined in terms of a biomarker or disease severity measured at baseline. The goal is to determine which treatments benefit which subpopulations. We develop a new class of adaptive enrichment designs tailored to solving this problem. Adaptive enrichment designs involve a preplanned rule for modifying enrollment based on accruing data in an ongoing trial. At the interim analysis after each stage, for each subpopulation, the preplanned rule may decide to stop enrollment or to stop randomizing participants to one or more study arms. The motivation for this adaptive feature is that interim data may indicate that a subpopulation, such as those with lower disease severity at baseline, is unlikely to benefit from a particular treatment while uncertainty remains for the other treatment and/or subpopulation. We optimize these adaptive designs to have the minimum expected sample size under power and Type I error constraints. We compare the performance of the optimized adaptive design versus an optimized nonadaptive (single stage) design. Our approach is demonstrated in simulation studies that mimic features of a completed trial of a medical device for treating heart failure. The optimized adaptive design has $25\%$ smaller expected sample size compared to the optimized nonadaptive design; however, the cost is that the optimized adaptive design has $8\%$ greater maximum sample size. Open-source software that implements the trial design optimization is provided, allowing users to investigate the tradeoffs in using the proposed adaptive versus standard designs.

We consider the problem of identifying subgroups of participants in a clinical trial that have enhanced treatment effect. Recursive partitioning methods that recursively partition the covariate space based on some measure of between groups treatment effect difference are popular for such subgroup identification. The most commonly used recursive partitioning method, the classification and regression tree algorithm, first creates a large tree by recursively partitioning the covariate space using some splitting criteria and then selects the final tree from all the subtrees of the large tree. In the context of subgroup identification, calculation of the splitting criteria and the evaluation measure used for final tree selection rely on comparing differences in means between the treatment and control arm. When covariates are prognostic for the outcome, covariate adjusted estimators have the ability to improve efficiency compared to using differences in averages between the treatment and control group. This manuscript develops two covariate adjusted estimators that can be used to both make splitting decisions and for final tree selection. The performance of the resulting covariate adjusted recursive partitioning algorithm is evaluated using simulations and by analyzing a clinical trial that evaluates if motivational interviews improve treatment engagement for substance abusers.

A crucial component of making individualized treatment decisions is to accurately predict each patient’s disease risk. In clinical oncology, disease risks are often measured through time-to-event data, such as overall survival and progression/recurrence-free survival, and are often subject to censoring. Risk prediction models based on recursive partitioning methods are becoming increasingly popular largely due to their ability to handle non-linear relationships, higher order interactions, and/or high dimensional covariates. The most popular recursive partitioning methods are versions of the Classification and Regression Tree (CART) algorithm, which builds a simple interpretable tree structured model. With the aim of increasing prediction accuracy, the random forest algorithm averages multiple CART trees, creating a flexible risk prediction model. Risk prediction models used in clinical oncology commonly use both traditional demographic and tumor pathological factors as well as high-dimensional genetic markers and treatment parameters from multi-modality treatments. In this article, we describe the most commonly used extensions of the CART and random forest algorithms to right-censored outcomes. We focus on how they differ from the methods for non-censored outcomes, and how the different splitting rules and methods for cost complexity pruning impact these algorithms. We demonstrate these algorithms by analyzing a randomized phase III clinical trial of breast cancer. We also conduct Monte Carlo simulations to compare the prediction accuracy of survival forests with more commonly used regression models under various scenarios. These simulation studies aim to evaluate how sensitive the prediction accuracy is to the underlying model specifications, the choice of tuning parameters, and the degrees of missing covariates.

–Standard computed tomography (CT) scan is performed on lung cancer patients for progression and lesion classification. However, low-dose CT (LDCT) is commonly used in lung cancer screening for high-risk people. Extensive studies have shown that computer-aided diagnosis (CAD) using standard CT could greatly improve the diagnostic accuracy of early lung cancer. Unlike standard CT imaging, the application of radiological texture features extracted by radiologists on LDCT imaging is not well established due to lower resolution and higher variations. The purpose of this study is to investigate possible diagnosis value of texture features by comparing the classification performance of radiologic reading with radiologic reading combined with computer-aided texture features. A total of 186 biopsy-confirmed control and lung cancer cases were obtained from the National Lung Screening Trial (NLST). Cases were matched by various clinical parameters including age, gender, smoking status, chronic obstructive pulmonary disease (COPD) status, body mass index (BMI) and image appearances. We compared the subjective diagnosis of benign/malignant with the consensus readings of three radiologists. We then developed a CAD framework that imports radiologic reading features and extracts CAD features for heterogeneity quantification and data analysis. A total of 1342 CAD features were extracted. After eliminating highly correlated and redundant features, the remaining 458 features were given to a random forest algorithm, and a predicted probability of malignancy score (Pm) was calculated. Patients were grouped into 140 training (70 biopsypositive for cancer and 70 negatives) and 46 testing (20 positives and 26 negatives) sets, and a threshold value over Pm (0.5) was then used to classify the test set into cancer and non-cancer. Clinical accuracy [sensitivity, specificity, positive predictive value (PPV), and negative predictive value (PV)] were [0.95, 0.88, 0.86, 0.96] and [0.70, 0.69, 0.64, 0.75] for the CAD and radiologic reading, respectively. The CAD framework incorporating the clinical reading with the texture features extracted from LDCT increased the PPV and reduced the false positive (FP) rate in the early diagnosis of lung cancer. This approach shows promise for improving the accuracy of lung cancer diagnosis in the clinical environment, especially in patients with well-established risk factors.

Increasing evidence suggests that persistence of Listeria monocytogenes in food processing plants has been the underlying cause of a number of human listeriosis outbreaks. This study extracts criteria used by food safety experts in determining bacterial persistence in the environment, using retail delicatessen operations as a model. Using the Delphi method, we conducted an expert elicitation with 10 food safety experts from academia, industry, and government to classify L. monocytogenes persistence based on environmental sampling results collected over six months for 30 retail delicatessen stores. The results were modeled using variations of random forest, support vector machine, logistic regression, and linear regression; variable importance values of random forest and support vector machine models were consolidated to rank important variables in the experts’ classifications. The duration of subtype isolation ranked most important across all expert categories. Sampling site category also ranked high in importance and validation errors doubled when this covariate was removed. Support vector machine and random forest models successfully classified the data with average validation errors of 3.1% and 2.2% (n = 144), respectively. Our findings indicate that (i) the frequency of isolations over time and sampling site information are critical factors for experts determining subtype persistence, (ii) food safety experts from different sectors may not use the same criteria in determining persistence, and (iii) machine learning models have potential for future use in environmental surveillance and risk management programs. Future work is necessary to validate the accuracy of expert and machine classification against biological measurement of L. monocytogenes persistence.

In randomized clinical trials with baseline variables that are prognostic for the primary outcome, there is potential to improve precision and reduce sample size by appropriately adjusting for these variables. A major challenge is that there are multiple statistical methods to adjust for baseline variables, but little guidance on which is best to use in a given context. The choice of method can have important consequences. For example, one commonly used method leads to uninterpretable estimates if there is any treatment effect heterogeneity, which would jeopardize the validity of trial conclusions. We give practical guidance on how to avoid this problem, while retaining the advantages of covariate adjustment. This can be achieved by using simple (but less well-known) standardization methods from the recent statistics literature. We discuss these methods and give software in R and Stata implementing them. A data example from a recent stroke trial is used to illustrate these methods.

Estimating a patient's mortality risk is important in making treatment decisions. Survival trees are a useful tool and employ recursive partitioning to separate patients into different risk groups. Existing 'loss based' recursive partitioning procedures that would be used in the absence of censoring have previously been extended to the setting of right censored outcomes using inverse probability censoring weighted estimators of loss functions. In this paper, we propose new 'doubly robust' extensions of these loss estimators motivated by semiparametric efficiency theory for missing data that better utilize available data. Simulations and a data analysis demonstrate strong performance of the doubly robust survival trees compared with previously used methods. Copyright © 2016 John WileySons, Ltd.