Search

Abstract Background Propensity scores are widely used to deal with confounding bias in medical research. An incorrectly specified propensity score model may lead to residual confounding bias; therefore it is essential to use diagnostics to assess propensity scores in a propensity score analysis. The current use of propensity score diagnostics in the medical literature is unknown. The objectives of this study are to (1) assess the use of propensity score diagnostics in medical studies published in high-ranking journals, and (2) assess whether the use of propensity score diagnostics differs between studies (a) in different research areas and (b) using different propensity score methods. Methods A PubMed search identified studies published in high-impact journals between Jan 1st 2014 and Dec 31st 2016 using propensity scores to answer an applied medical question. From each study we extracted information regarding how propensity scores were assessed and which propensity score method was used. Research area was defined using the journal categories from the Journal Citations Report. Results A total of 894 papers were included in the review. Of these, 187 (20.9%) failed to report whether the propensity score had been assessed. Commonly reported diagnostics were p-values from hypothesis tests (36.6%) and the standardised mean difference (34.6%). Statistical tests provided marginally stronger evidence for a difference in diagnostic use between studies in different research areas (p = 0.033) than studies using different propensity score methods (p = 0.061). Conclusions The use of diagnostics in the propensity score medical literature is far from optimal, with different diagnostics preferred in different areas of medicine. The propensity score literature may improve with focused efforts to change practice in areas where suboptimal practice is most common.

This paper presents an analysis of survey data to examine the association between participating in one of the National Institute of Health's (NIH) funded Building Infrastructure Leading to Diversity Initiative (BUILD) program and students' intent to pursue a career in science. Data were collected from students at California State University Long Beach (CSULB) to examine the effectiveness of the BUILD Scholars program. Both BUILD Scholars and non-BUILD students were surveyed. Propensity score matching was used to generate the non-BUILD comparison group. Multinomial logistic regression results revealed that students participating in the BUILD intervention were associated with significantly higher intent to pursue a career in science. Results also showed the importance of variables such as science identity and research participation when assessing interest in science careers. These findings have implications for STEM program evaluation and practice in higher education. [ABSTRACT FROM AUTHOR]

Sampling is informative when probabilities of sample inclusion depend on unknown variables that are correlated with a response variable of interest. When sample inclusion probabilities are available, inverse probability weighting can be used to account for informative sampling in such a situation, although usually at the cost of less precise inference. This paper reviews two important research contributions by Chris Skinner that modify these weights to reduce their variability while at the same time retaining consistency of the weighted estimators. In some cases, however, sample inclusion probabilities are not known, and are estimated as propensity scores. This is often the situation in causal analysis, and double robust methods that protect against the resulting misspecification of the sampling process have been the focus of much recent research. In this paper we propose two model‐assisted modifications to the popular inverse propensity score weighted estimator of an average treatment effect, and then illustrate their use in a causal analysis of a rainfall enhancement experiment that was carried out in Oman between 2013 and 2018. [ABSTRACT FROM AUTHOR]

Propensity score methods are popular to control for confounding in observational biomedical studies of risk factors or medical treatments. This paper focused on aspects of propensity score methods that often remain undiscussed, including unmeasured confounding, missing data, variable selection, statistical efficiency, estimands, the positivity assumption, and predictive performance of the propensity score model. [ABSTRACT FROM AUTHOR]

An equating procedure for a testing program with evolving distribution of examinee profiles is developed. No anchor is available because the original scoring scheme was based on expert judgment of the item difficulties. Pairs of examinees from two administrations are formed by matching on coarsened propensity scores derived from a set of background variables. These two subsets of scores are then equated, treating the associated sets of test performances as equivalent. The method is applied to the scores in 2 years of a testing program for admission to tertiary education. [ABSTRACT FROM AUTHOR]

AbstractThere has been a recent surge in statistical methods for handling the lack of adequate positivity when using inverse probability weights (IPW). However, these nascent developments have raised a number of questions. Thus, we demonstrate the ability of equipoise estimators (overlap, matching, and entropy weights) to handle the lack of positivity. Compared to IPW, the equipoise estimators have been shown to be flexible and easy to interpret. However, promoting their wide use requires that researchers know clearly why, when to apply them and what to expect. In this paper, we provide the rationale to use these estimators to achieve robust results. We specifically look into the impact imbalances in treatment allocation can have on the positivity and, ultimately, on the estimates of the treatment effect. We zero into the typical pitfalls of the IPW estimator and its relationship with the estimators of the average treatment effect on the treated (ATT) and on the controls (ATC). Furthermore, we also compare IPW trimming to the equipoise estimators. We focus particularly on two key points: What fundamentally distinguishes their estimands? When should we expect similar results? Our findings are illustrated through Monte-Carlo simulation studies and a data example on healthcare expenditure. [ABSTRACT FROM AUTHOR]

In certain indications, it is well understood that randomized controlled trials lead to slow enrollment and high differential drop-out rate in the standard of care control arm when the standard of care is undesirable to patients. This not only impacts the pace of drug development but may also render a randomized trial uninterpretable if drop-out from the control arm is common. Single-arm trials are common in such indications. An external control arm (ECA) built using a propensity score method (Rosenbaum and Rubin, Biometrika 70:41–55, 1983) from subjects outside the current trial but who meet the same eligibility criteria as the subjects of the current trial, is valuable in assessing treatment effect of a new drug that cannot be otherwise assessed with a single-arm trial, or an unintentionally under-powered and arguably biased randomized controlled trial. Propensity score methods have increasingly gained importance in observational medical research, paving the way for the use of these methods in non-randomized clinical trials. In this paper, we describe our experience with building an ECA for drug development related to regulatory activities associated with a new molecular entity (NME). Through analysis of a phase 2 study, we show how best practices from causal statistical methods can be used to build an ECA for a non-randomized drug trial—a novel application of a well-studied method. To prevent selection bias, the selection of subjects for the ECA was carried out by an independent group of statisticians who were blinded to all patient outcome data. We share lessons learnt from our regulatory interactions that provide helpful suggestions on how to advance ECA-based drug development. There has been interest in confirmatory trials that use reduced control arm randomization along with an ECA. We present a design for such a hybrid trial using a propensity score weighting method and describe how a standard propensity score analysis can be used to analyze data resulting from a composite hybrid randomized and external control arm. [ABSTRACT FROM AUTHOR]

Deprescribing is defined as "the planned and supervised process of dose reduction or stopping of medication that might be causing harm, or no longer be of benefit". Barriers to deprescribing include healthcare professional fear and lack of guidance. These may stem from limited available evidence on benefits and harms of deprescribing medications commonly used among older persons. Advances in pharmacoepidemiology and causal inference methods to evaluate comparative effectiveness and safety of prescribing medications have yet to be considered for deprescribing medication. This paper discusses select methods and how they can be applied to deprescribing research, using case studies of benzodiazepines and low-dose acetylsalicylic acid (aspirin). Target trial emulation involves the explicit application of design principles from randomised controlled trials to observational studies. Several design aspects, including defining eligibility criteria and time zero, require additional considerations for deprescribing studies. The active comparator new user design also presents challenges, including selection of an appropriate comparator. This paper discusses these aspects, and others, in relation to deprescribing studies. Furthermore, methods proposed to control for confounding, in particular, the prior event rate ratio and propensity scores, are discussed. Introduction of billing codes or mechanisms for accurately determining when deprescribing has occurred would enhance the ability to conduct research using routinely collected data. Although the approaches discussed in this paper may strengthen observational studies of deprescribing, their use may be best suited to certain scenarios or research questions, where randomised controlled trials may be less feasible. [ABSTRACT FROM AUTHOR]

Summary: The scientific rigor and computational methods of causal inference have had great impacts on many disciplines but have only recently begun to take hold in spatial applications. Spatial causal inference poses analytic challenges due to complex correlation structures and interference between the treatment at one location and the outcomes at others. In this paper, we review the current literature on spatial causal inference and identify areas of future work. We first discuss methods that exploit spatial structure to account for unmeasured confounding variables. We then discuss causal analysis in the presence of spatial interference including several common assumptions used to reduce the complexity of the interference patterns under consideration. These methods are extended to the spatiotemporal case where we compare and contrast the potential outcomes framework with Granger causality and to geostatistical analyses involving spatial random fields of treatments and responses. The methods are introduced in the context of observational environmental and epidemiological studies and are compared using both a simulation study and analysis of the effect of ambient air pollution on COVID‐19 mortality rate. Code to implement many of the methods using the popular Bayesian software OpenBUGS is provided. [ABSTRACT FROM AUTHOR]

Recent years have seen increasing interest in and attention to evidence-based practices, where the 'evidence' generally comes from well-conducted randomized trials. However, while those trials yield accurate estimates of the effect of the intervention for the participants in the trial (known as 'internal validity'), they do not always yield relevant information about the effects in a particular target population (known as 'external validity'). This may be due to a lack of specification of a target population when designing the trial, difficulties recruiting a sample that is representative of a prespecified target population, or to interest in considering a target population somewhat different from the population directly targeted by the trial. This paper first provides an overview of existing design and analysis methods for assessing and enhancing the ability of a randomized trial to estimate treatment effects in a target population. It then provides a case study using one particular method, which weights the subjects in a randomized trial to match the population on a set of observed characteristics. The case study uses data from a randomized trial of school-wide positive behavioral interventions and supports (PBIS); our interest is in generalizing the results to the state of Maryland. In the case of PBIS, after weighting, estimated effects in the target population were similar to those observed in the randomized trial. The paper illustrates that statistical methods can be used to assess and enhance the external validity of randomized trials, making the results more applicable to policy and clinical questions. However, there are also many open research questions; future research should focus on questions of treatment effect heterogeneity and further developing these methods for enhancing external validity. Researchers should think carefully about the external validity of randomized trials and be cautious about extrapolating results to specific populations unless they are confident of the similarity between the trial sample and that target population. [ABSTRACT FROM AUTHOR]

School-based evaluations of interventions are increasingly common in education research. Ideally, the results of these evaluations are used to make evidence-based policy decisions for students. However, it is difficult to make generalizations from these evaluations because the types of schools included in the studies are typically not selected randomly from a target population. This paper provides an overview of statistical methods for improving generalizations from intervention research in education. These are presented as a series of steps aimed at improving research design—particularly recruitment—as well as methods for assessing and summarizing generalizability and estimating treatment impacts for clearly defined target populations. [ABSTRACT FROM AUTHOR]

This paper considers the estimation of average treatment effect using propensity score method. We propose to use a semiparametric single-index model to estimate the propensity score. This avoids the curse of dimensionality problem with the nonparametric method based propensity score estimator. We establish the asymptotic distribution of the average treatment effect estimator. Monte Carlo simulation results show that the proposed method works well in finite samples and outperforms the conventional nonparametric kernel approach. We apply the proposed method to an empirical data examining the efficacy of right heart catheterization on medical outcomes. [ABSTRACT FROM AUTHOR]

The current kidney allocation system in the United States fails to match donors and recipients well. In an effort to improve the allocation system, the United Network of Organ Sharing (UNOS) defined factors that should determine a new allocation policy, and particularly patients' potential remaining lifetime without a transplant (pre-transplant survival rates). Estimating pre-transplant survival rates is challenging because the data available on candidates and organ recipients is already 'contaminated' by the current allocation policy. In particular, the selection of patients who are offered (and decide to accept) a kidney is not random. We therefore expect differences in mortality-related characteristics of organ recipients and of candidates who have not received transplant. Such differences introduce bias into survival models. Existing approaches for tackling this selection bias either ignore the difference between candidates and recipients or assume that selection to transplant is based solely on patients' pre-transplant information, irrespective of the potential allocation outcome. We argue that in practice the allocation is dependent on the anticipated allocation outcome, which is a major factor in selection to transplant. Moreover, we show that ignoring the anticipated outcome increases model bias rather than decreases it. In this paper, we propose a novel simulator-based approach (SimBa) that adjusts for the selection bias by taking into account both pre-transplant and anticipated outcome information using simulation. We then fit survival models to kidney transplant waitlist data and compare the different adjustment methods. We find that SimBa not only fits the data best, but also captures a key aspect of the current allocation policy, namely, that the probability of kidney allocation increases in the expected pre-transplant life years. [ABSTRACT FROM AUTHOR]

Many lifestyle intervention trials depend on collecting self-reported outcomes, such as dietary intake, to assess the intervention's effectiveness. Self-reported outcomes are subject to measurement error, which impacts treatment effect estimation. External validation studies measure both self-reported outcomes and accompanying biomarkers, and can be used to account for measurement error. However, in order to account for measurement error using an external validation sample, an assumption must be made that the inferences are transportable from the validation sample to the intervention trial of interest. This assumption does not always hold. In this paper, we propose an approach that adjusts the validation sample to better resemble the trial sample, and we also formally investigate when bias due to poor transportability may arise. Lastly, we examine the performance of the methods using simulation, and illustrate them using PREMIER, a lifestyle intervention trial measuring self-reported sodium intake as an outcome, and OPEN, a validation study measuring both self-reported diet and urinary biomarkers. [ABSTRACT FROM AUTHOR]

Joint misclassification of exposure and outcome variables can lead to considerable bias in epidemiological studies of causal exposure-outcome effects. In this paper, we present a new maximum likelihood based estimator for marginal causal effects that simultaneously adjusts for confounding and several forms of joint misclassification of the exposure and outcome variables. The proposed method relies on validation data for the construction of weights that account for both sources of bias. The weighting estimator, which is an extension of the outcome misclassification weighting estimator proposed by Gravel and Platt (Weighted estimation for confounded binary outcomes subject to misclassification. Stat Med 2018; 37: 425–436), is applied to reinfarction data. Simulation studies were carried out to study its finite sample properties and compare it with methods that do not account for confounding or misclassification. The new estimator showed favourable large sample properties in the simulations. Further research is needed to study the sensitivity of the proposed method and that of alternatives to violations of their assumptions. The implementation of the estimator is facilitated by a new R function (ipwm) in an existing R package (mecor). [ABSTRACT FROM AUTHOR]

There are two seemingly unrelated approaches to weighting in observational studies. One of them maximizes the fit of a model for treatment assignment to then derive weights—we call this the modeling approach. The other directly optimizes certain features of the weights—we call this the balancing approach. The implementations of these two approaches are related: the balancing approach implicitly models the propensity score, while instances of the modeling approach impose balance conditions on the covariates used to estimate the propensity score. In this article, we review and compare these two approaches to weighting. Previous review papers have focused on the modeling approach, emphasizing the importance of checking covariate balance. However, as we discuss, the dispersion of the weights is another important aspect of the weights to consider, in addition to the representativeness of the weighted sample and the sample boundedness of the weighted estimator. In particular, the dispersion of the weights is important because it translates into a measure of effective sample size, which can be used to select between alternative weighting schemes. In this article, we examine the balancing approach to weighting, discuss recent methodological developments, and compare instances of the balancing and modeling approaches in a simulation study and an empirical study. In practice, unless the treatment assignment model is known, we recommend using the balancing approach to weighting, as it systematically results in better covariate balance with weights that are minimally dispersed. As a result, effect estimates tend to be more accurate and stable. [ABSTRACT FROM AUTHOR]

In this paper, a propensity score-integrated composite likelihood (PSCL) approach is developed for cases in which the control arm of a two-arm randomized controlled trial (RCT) (treated vs control) is augmented with patients from real-world data (RWD) containing both clinical outcomes and covariates at the patient-level. RWD patients who were treated with the same therapy as the control arm of the RCT are considered for the augmentation. The PSCL approach first estimates the propensity score for every patient as the probability of the patient being in the RCT rather than the RWD, and then stratifies all patients into strata based on the estimated propensity scores. Within each propensity score stratum, a composite likelihood function is specified and utilized to down-weight the information contributed by the RWD source. Estimates of the stratum-specific parameters are obtained by maximizing the composite likelihood function. These stratum-specific estimates are then combined to obtain an overall population-level estimate of the parameter of interest. The performance of the proposed approach is evaluated via a simulation study. A hypothetical two-arm RCT and a hypothetical RWD source are used to illustrate the implementation of the proposed approach. [ABSTRACT FROM AUTHOR]

Randomized controlled trials are not always feasible in educational research, so researchers must use alternative methods to study treatment effects. Propensity score matching is one such method for observational studies that has shown considerable growth in popularity since it was first introduced in the early 1980s. This paper outlines the concept of propensity scores by explaining their theoretical principles and providing two examples of their usefulness within the realm of educational research. Through worked examples, we highlight the effectiveness of propensity scores as a method for reducing bias and increasing the balance between treatment and comparison groups. To aid in the understanding and future use of propensity scores, we provide R syntax for all our analyses. [ABSTRACT FROM AUTHOR]

This paper builds upon two prior papers by Haviland and Nagin (Psychometrika 70:1–22, 2005) and Haviland, Nagin, and Rosenbaum (Working paper, 2006) that attempt to bring the key attributes of an experiment to the analysis of non-experimental longitudinal data. Using a case study of the facilitation effect of gang membership on violence, it systematically examines the contribution of group-based trajectory modeling to the achievement of covariate balance in observational data. In this case study, inclusion of the posterior probabilities of group membership (PPGM), from a model on the pre-treatment measures of the outcome variable, created closer balance on these key covariates than did analyses that did not include them. Still closer balance was obtained on these key covariates by stratifying the analysis by trajectory group. This stratification was achieved by fitting separate propensity score models and matching gang joiners to gang abstainers within trajectory group. In addition, we demonstrated that further balance could be obtained on additional covariates by including PPGM from a model on pre-treatment longitudinal data of these covariates. While this case study is only one empirical example, we believe that it provides useful empirical evidence on the value of performing within trajectory group causal inference in observational longitudinal data and on the use of the PPGM in achieving balance in propensity score-based causal inference. [ABSTRACT FROM AUTHOR]

Background: Recently, there has been a heightened interest in developing and evaluating different methods for analysing observational data. This has been driven by the increased availability of large data resources such as Electronic Health Record (EHR) data alongside known limitations and changing characteristics of randomised controlled trials (RCTs). A wide range of methods are available for analysing observational data. However, various, sometimes strict, and often unverifiable assumptions must be made in order for the resulting effect estimates to have a causal interpretation. In this paper we will compare some common approaches to estimating treatment effects from observational data in order to highlight the importance of considering, and justifying, the relevant assumptions prior to conducting an observational analysis.Methods: A simulation study was conducted based upon a small cohort of patients with chronic obstructive pulmonary disease. Two-stage least squares instrumental variables, propensity score, and linear regression models were compared under a range of different scenarios including different strengths of instrumental variable and unmeasured confounding. The effects of violating the assumptions of the instrumental variables analysis were also assessed. Sample sizes of up to 200,000 patients were considered.Results: Two-stage least squares instrumental variable methods can yield unbiased treatment effect estimates in the presence of unmeasured confounding provided the sample size is sufficiently large. Adjusting for measured covariates in the analysis reduces the variability in the two-stage least squares estimates. In the simulation study, propensity score methods produced very similar results to linear regression for all scenarios. A weak instrument or strong unmeasured confounding led to an increase in uncertainty in the two-stage least squares instrumental variable effect estimates. A violation of the instrumental variable assumptions led to bias in the two-stage least squares effect estimates. Indeed, these were sometimes even more biased than those from a naïve linear regression model.Conclusions: Instrumental variable methods can perform better than naïve regression and propensity scores. However, the assumptions need to be carefully considered and justified prior to conducting an analysis or performance may be worse than if the problem of unmeasured confounding had been ignored altogether. [ABSTRACT FROM AUTHOR]

Observational studies are commonplace in medicine. A frequent concern is confounding bias due to differences in patient characteristics across treatment groups, but other important issues include dependency among observations nested within clusters (e.g. patients clustered within physicians or surgeons) and confounding due to cluster characteristics (e.g. physician or surgeon experience or training). Given the frequency with which these issues arise in medical research, as well as their relative complexity, methods for the analysis of clustered observational data are reviewed. We argue for estimating causal treatment effects using marginal models that either match or weight observations using a suitable distance metric (e.g. the propensity score). Simulation results demonstrated that methods incorporating clustering into calculation of the variance were generally more accurate than those that did not. Moreover, methods that account for cluster confounding when estimating the treatment effect were least biased and most accurate. Throughout the paper we illustrate the proposed methods in a medical device setting that compares the effectiveness of femoral heads used in total hip replacements. Whenever possible the clustered aspect of the data should be considered in the design of the study when constructing the distance measure or in the matching process, as well as in the analysis when estimating the variance of the treatment effect. [ABSTRACT FROM AUTHOR]

One important goal in pharmaco-epidemiology studies is to understand the causal relationship between drug exposures and their clinical outcomes, including adverse drug events. In order to achieve this goal, however, we need to resolve several challenges. Most of pharmaco-epidemiology data are observational and confounding is largely present due to many co-medications. The pharmaco-epidemiology study data set is often sampled from large medical record databases using a matched case-control design, and it may not be representative of the original patient population in the medical record databases. Data analysis method needs to handle a large sample size that cannot be handled using existing statistical analysis packages. In this paper, we tackle these challenges both methodologically and computationally. We propose a conditional causal log-odds ratio (OR) definition to characterize causal effects of drug exposures on a binary adverse drug event adjusting for individual level confounders. Using a case-control design, we present a propensity score estimation using only case samples and we provide sufficient conditions for the consistency of the estimation of the causal log-odds ratio using case-based propensity scores. Computationally, we implement a principle component analysis to reduce high-dimensional confounders. Extensive simulation studies are performed to demonstrate superior performance of our method to existing methods. Finally, we apply the proposed method to analyze drug-induced myopathy data sampled from a de-identified subset of medical record database (close to 5 million patient records), The Indiana Network for Patient Care. Our method identified 70 drug-induced myopathy (p < 0.05) out 72 drugs, which have myoathy side effects on their FDA drug labels. These 70 drugs include three statins who are known for their myopathy side effects. [ABSTRACT FROM AUTHOR]

The evaluation of policies that are not randomly assigned on outcomes generated by nonlinear data generating processes often requires modeling assumptions for which there is little theoretical guidance. This paper revisits previously published difference-in-differences results of an important example, the introduction of reference pricing to common outpatient procedures, to assess the robustness of the estimated impacts by using different matching, and reweighting techniques to preprocess the data. These techniques improve covariate balance and reduce model dependence. Specifically, we examine the robustness of the effect of reference pricing on patient site-of-care choice, total expenditures, and complication rates. We apply three preprocessing methods: propensity score reweighting, exact matching, and genetic matching. Propensity score reweighting is a technique for achieving covariate balance but does not balance higher-order moments and may lead to bias and inefficiency in estimating treatment effects in the context of nonlinear data generating processes. In contrast, exact matching and genetic matching are designed to balance higher-order moments. We find that although the use of the preprocessing techniques is a valuable robustness check showing that some results are sensitive to the method used, the three approaches generally yield results that do not statistically differ from the published results. [ABSTRACT FROM AUTHOR]

Many approaches, including traditional parametric modeling and machine learning techniques, have been proposed to estimate propensity scores. This paper describes a new model averaging approach to propensity score estimation in which parametric and nonparametric estimates are combined to achieve covariate balance. Simulation studies are conducted across different scenarios varying in the degree of interactions and nonlinearities in the treatment model. The results show that, based on inverse probability weighting, the proposed propensity score estimator produces less bias and smaller standard errors than existing approaches. They also show that a model averaging approach with the objective of minimizing the average Kolmogorov-Smirnov statistic leads to the best performing IPW estimator. The proposed approach is also applied to a real data set in evaluating the causal effect of formula or mixed feeding versus exclusive breastfeeding on a child's body mass index Z-score at age 4. The data analysis shows that formula or mixed feeding is more likely to lead to obesity at age 4, compared to exclusive breastfeeding. [ABSTRACT FROM AUTHOR]

Whether work is performed for household members’ consumption (subsistence work) or for sale to others (market work), it may be an enabling resource for women’s agency, or their capacity to define and act upon their goals. The present paper asks: Do women who engage in market work have higher agency in the three domains of economic decision-making, freedom of movement, and equitable gender role attitudes, compared to those who engage in subsistence work and those who do not work? To address this question, we leverage data from a probability sample of ever-married women in rural Minya, Egypt (N = 600). We use structural equation models with propensity score adjustment to estimate the relationship between women’s work and three domains of their agency. We find no effect on gender attitudes or decision making. However, women’s subsistence and market work are associated with increasingly higher factor means for freedom of movement, compared to not working. We conclude that in rural Minya, the relationship between women’s work and their agency depends on the type of work they perform and the dimension of agency under consideration, with the rewards of market work exceeding those of subsistence work in the domain of freedom of movement alone. [ABSTRACT FROM AUTHOR]

Distributed networks of health-care data sources are increasingly being utilized to conduct pharmacoepidemio-logic database studies. Such networks may contain data that are not physically pooled but instead are distributed horizontally (separate patients within each data source) or vertically (separate measures within each data source) in order to preserve patient privacy. While multivariable methods for the analysis of horizontally distributed data are frequently employed, few practical approaches have been put forth to deal with vertically distributed healthcare databases. In this paper, we propose 2 propensity score-based approaches to vertically distributed data analysis and test their performance using 5 example studies. We found that these approaches produced point estimates close to what could be achieved without partitioning. We further found a performance benefit (i.e., lower mean squared error) for sequentially passing a propensity score through each data domain (called the "sequential approach") as compared with fitting separate domain-specific propensity scores (called the "parallel approach"). These results were validated in a small simulation study. This proof-of-concept study suggests a new multivariable analysis approach to vertically distributed health-care databases that is practical, preserves patient privacy, and warrants further investigation for use in clinical research applications that rely on health-care databases. [ABSTRACT FROM AUTHOR]

This study examined the validity evidence of automated scores across English learners (ELs) and non-EL test takers in a statewide summative writing assessment. Writing performance for the two groups in online and paper-and-pencil formats was analyzed using operational data. Online essays were scored by an automated scoring engine and handwritten essays were scored by humans. Differential item functioning (DIF) analyses and propensity score matching (PSM) methods were conducted to examine any systematic difference between the two groups. DIF results indicated that ELs scored higher than expected on paper in Grades 10 and 11, as compared to non-ELs. In addition, DIF analyses comparing EL performance online versus paper demonstrated DIF favoring paper in upper grades. After controlling for student characteristics, differences still remained and the effect sizes were only reduced slightly. Follow-up analyses on the matched samples revealed similar levels of DIF as compared to the unmatched samples. These findings have important implications in interpreting writing scores for ELs. Recommendations for the modification of scoring and training protocols for human raters and automated scoring systems to better understand the relationship between human and machine scored essays are also provided. • Paper format, which was scored by human raters, tended to favor English learners. • The majority of effect size came from the lower end of the distribution. • No significant difference was observed between ELs and non-ELs with online sample. [ABSTRACT FROM AUTHOR]

Causal inference analyses often use existing observational data, which in many cases has some clustering of individuals. In this paper, we discuss propensity score weighting methods in a multilevel setting where within clusters individuals share unmeasured confounders that are related to treatment assignment and the potential outcomes. We focus in particular on settings where models with fixed cluster effects are either not feasible or not useful due to the presence of a large number of small clusters. We found, both through numerical experiments and theoretical derivations, that a strategy of grouping clusters with similar treatment prevalence and estimating propensity scores within such cluster groups is effective in reducing bias from unmeasured cluster‐level covariates under mild conditions on the outcome model. We apply our proposed method in evaluating the effectiveness of centre‐based pre‐school programme participation on children's achievement at kindergarten, using the Early Childhood Longitudinal Study Kindergarten data. [ABSTRACT FROM AUTHOR]

This paper conducts a Monte Carlo simulation study to evaluate the performance of multivariate matching methods that select a subset of treatment and control observations. The matching methods studied are the widely used nearest neighbor matching with propensity score calipers and the more recently proposed methods, optimal matching of an optimally chosen subset and optimal cardinality matching. The main findings are: (i) covariate balance, as measured by differences in means, variance ratios, Kolmogorov-Smirnov distances, and cross-match test statistics, is better with cardinality matching because by construction it satisfies balance requirements; (ii) for given levels of covariate balance, the matched samples are larger with cardinality matching than with the other methods; (iii) in terms of covariate distances, optimal subset matching performs best; (iv) treatment effect estimates from cardinality matching have lower root-mean-square errors, provided strong requirements for balance, specifically, fine balance, or strength-k balance, plus close mean balance. In standard practice, a matched sample is considered to be balanced if the absolute differences in means of the covariates across treatment groups are smaller than 0.1 standard deviations. However, the simulation results suggest that stronger forms of balance should be pursued in order to remove systematic biases due to observed covariates when a difference in means treatment effect estimator is used. In particular, if the true outcome model is additive, then marginal distributions should be balanced, and if the true outcome model is additive with interactions, then low-dimensional joints should be balanced. Copyright © 2016 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

A variety of different study designs and analysis methods have been used to evaluate the performance of traffic safety countermeasures. The most common study designs and methods include observational before–after studies using the empirical Bayes method and cross-sectional studies using regression models. The propensity scores-potential outcomes framework has recently been proposed as an alternative traffic safety countermeasure evaluation method to address the challenges associated with selection biases that can be part of cross-sectional studies. Crash modification factors derived from the application of all three methods have not yet been compared. This paper compares the results of retrospective, observational evaluations of a traffic safety countermeasure using both before–after and cross-sectional study designs. The paper describes the strengths and limitations of each method, focusing primarily on how each addresses site selection bias, which is a common issue in observational safety studies. The Safety Edge paving technique, which seeks to mitigate crashes related to roadway departure events, is the countermeasure used in the present study to compare the alternative evaluation methods. The results indicated that all three methods yielded results that were consistent with each other and with previous research. The empirical Bayes results had the smallest standard errors. It is concluded that the propensity scores with potential outcomes framework is a viable alternative analysis method to the empirical Bayes before–after study. It should be considered whenever a before–after study is not possible or practical. [ABSTRACT FROM AUTHOR]

There is an increasing interest in the use of stated preference methods to understand individuals' preferences for health and healthcare. There is also a growing interest in understanding heterogeneity in individuals' preferences. Consequently, stated preference studies frequently consider models that capture either or both observed and unobserved preference heterogeneity. A popular preliminary investigation into heterogeneity involves split-sample analysis to compare subgroups' preferences e.g., comparing patients with clinicians, or older patients with younger. In fixed-effects models, the constant variables (the individuals' characteristics) remain stable across choice sets and therefore only enter the choice model when interacted with various attributes and/or levels. However, subgroups of respondents may differ on multiple variables that may not easily be implemented with interaction terms because of complexity and a lack of power thus only one, or a few, variables are typically taken into account in each subgroup model. This paper presents an overview of methods for matching and balancing samples to weight individuals with different characteristics in subgroup analysis and an example of how unweighted comparisons may produce erroneous conclusions regarding the degree of heterogeneity in preferences. We illustrate the issue with synthetic and empirical datasets to explore methods for matching subgroups before and within simple choice models. Our results show that entropy balancing and propensity score matching could be more appropriate than analyses using unmatched preference data when heterogeneity is driven by multiple factors. The paper concludes with a discussion of when matching and weighting may and may not be useful in healthcare decision-making. • Subgroup analysis to investigate preference heterogeneity is commonplace. • Direct subgroup comparisons can be biased if other characteristics also influence preferences. • Matching or weighting techniques can be used to ensure balance of multiple characteristics across subgroups. • Whether these techniques are required depends on the research question of interest. [ABSTRACT FROM AUTHOR]

Despite growing recognition of the important role implementation plays in successful prevention efforts, relatively little work has sought to demonstrate a causal relationship between implementation factors and participant outcomes. In turn, failure to explore the implementation-to-outcome link limits our understanding of the mechanisms essential to successful programming. This gap is partially due to the inability of current methodological procedures within prevention science to account for the multitude of confounders responsible for variation in implementation factors (i.e., selection bias). The current paper illustrates how propensity and marginal structural models can be used to improve causal inferences involving implementation factors not easily randomized (e.g., participant attendance). We first present analytic steps for simultaneously evaluating the impact of multiple implementation factors on prevention program outcome. Then, we demonstrate this approach for evaluating the impact of enrollment and attendance in a family program, over and above the impact of a school-based program, within PROSPER, a large-scale real-world prevention trial. Findings illustrate the capacity of this approach to successfully account for confounders that influence enrollment and attendance, thereby more accurately representing true causal relations. For instance, after accounting for selection bias, we observed a 5 % reduction in the prevalence of 11th grade underage drinking for those who chose to receive a family program and school program compared to those who received only the school program. Further, we detected a 7 % reduction in underage drinking for those with high attendance in the family program. [ABSTRACT FROM AUTHOR]

Confounding present in observational data impede community psychologists' ability to draw causal inferences. This paper describes propensity score methods as a conceptually straightforward approach to drawing causal inferences from observational data. A step-by-step demonstration of three propensity score methods-weighting, matching, and subclassification-is presented in the context of an empirical examination of the causal effect of preschool experiences (Head Start vs. parental care) on reading development in kindergarten. Although the unadjusted population estimate indicated that children with parental care had substantially higher reading scores than children who attended Head Start, all propensity score adjustments reduce the size of this overall causal effect by more than half. The causal effect was also defined and estimated among children who attended Head Start. Results provide no evidence for improved reading if those children had instead received parental care. We carefully define different causal effects and discuss their respective policy implications, summarize advantages and limitations of each propensity score method, and provide SAS and R syntax so that community psychologists may conduct causal inference in their own research. [ABSTRACT FROM AUTHOR]

ABSTRACT This paper addresses strategies for selecting variables for adjustment in non-experimental comparative effectiveness research and uses causal graphs to illustrate the causal network that relates treatment to outcome. Variables in the causal network take on multiple structural forms. Adjustment for a common cause pathway between treatment and outcome can remove confounding, whereas adjustment for other structural types may increase bias. For this reason, variable selection would ideally be based on an understanding of the causal network; however, the true causal network is rarely known. Therefore, we describe more practical variable selection approaches based on background knowledge when the causal structure is only partially known. These approaches include adjustment for all observed pretreatment variables thought to have some connection to the outcome, all known risk factors for the outcome, and all direct causes of the treatment or the outcome. Empirical approaches, such as forward and backward selection and automatic high-dimensional proxy adjustment, are also discussed. As there is a continuum between knowing and not knowing the causal, structural relations of variables, we recommend addressing variable selection in a practical way that involves a combination of background knowledge and empirical selection and that uses high-dimensional approaches. This empirical approach can be used to select from a set of a priori variables based on the researcher's knowledge to be included in the final analysis or to identify additional variables for consideration. This more limited use of empirically derived variables may reduce confounding while simultaneously reducing the risk of including variables that may increase bias. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

In this paper, we address the Bayesian classification with incomplete data. The common approach in the literature is to simply ignore the samples with missing values or impute missing values before classification. However, these methods are not effective when a large portion of the data have missing values and the acquisition of samples is expensive. Motivated by these limitations, the expectation maximization algorithm for learning a multivariate Gaussian mixture model and a multiple kernel density estimator based on the propensity scores are proposed to avoid listwise deletion (LD) or mean imputation (MI) for solving classification tasks with incomplete data. We illustrate the effectiveness of our proposed algorithms on some artificial and benchmark UCI data sets by comparing with LD and MI methods. We also apply these algorithms to solve the practical classification tasks on the lithology identification of hydrothermal minerals and license plate character recognition. The experimental results demonstrate their good performance with high classification accuracies. [ABSTRACT FROM AUTHOR]