Search

An equating procedure for a testing program with evolving distribution of examinee profiles is developed. No anchor is available because the original scoring scheme was based on expert judgment of the item difficulties. Pairs of examinees from two administrations are formed by matching on coarsened propensity scores derived from a set of background variables. These two subsets of scores are then equated, treating the associated sets of test performances as equivalent. The method is applied to the scores in 2 years of a testing program for admission to tertiary education. [ABSTRACT FROM AUTHOR]

School-based evaluations of interventions are increasingly common in education research. Ideally, the results of these evaluations are used to make evidence-based policy decisions for students. However, it is difficult to make generalizations from these evaluations because the types of schools included in the studies are typically not selected randomly from a target population. This paper provides an overview of statistical methods for improving generalizations from intervention research in education. These are presented as a series of steps aimed at improving research design—particularly recruitment—as well as methods for assessing and summarizing generalizability and estimating treatment impacts for clearly defined target populations. [ABSTRACT FROM AUTHOR]

Many lifestyle intervention trials depend on collecting self-reported outcomes, such as dietary intake, to assess the intervention's effectiveness. Self-reported outcomes are subject to measurement error, which impacts treatment effect estimation. External validation studies measure both self-reported outcomes and accompanying biomarkers, and can be used to account for measurement error. However, in order to account for measurement error using an external validation sample, an assumption must be made that the inferences are transportable from the validation sample to the intervention trial of interest. This assumption does not always hold. In this paper, we propose an approach that adjusts the validation sample to better resemble the trial sample, and we also formally investigate when bias due to poor transportability may arise. Lastly, we examine the performance of the methods using simulation, and illustrate them using PREMIER, a lifestyle intervention trial measuring self-reported sodium intake as an outcome, and OPEN, a validation study measuring both self-reported diet and urinary biomarkers. [ABSTRACT FROM AUTHOR]

Joint misclassification of exposure and outcome variables can lead to considerable bias in epidemiological studies of causal exposure-outcome effects. In this paper, we present a new maximum likelihood based estimator for marginal causal effects that simultaneously adjusts for confounding and several forms of joint misclassification of the exposure and outcome variables. The proposed method relies on validation data for the construction of weights that account for both sources of bias. The weighting estimator, which is an extension of the outcome misclassification weighting estimator proposed by Gravel and Platt (Weighted estimation for confounded binary outcomes subject to misclassification. Stat Med 2018; 37: 425–436), is applied to reinfarction data. Simulation studies were carried out to study its finite sample properties and compare it with methods that do not account for confounding or misclassification. The new estimator showed favourable large sample properties in the simulations. Further research is needed to study the sensitivity of the proposed method and that of alternatives to violations of their assumptions. The implementation of the estimator is facilitated by a new R function (ipwm) in an existing R package (mecor). [ABSTRACT FROM AUTHOR]

Observational studies are commonplace in medicine. A frequent concern is confounding bias due to differences in patient characteristics across treatment groups, but other important issues include dependency among observations nested within clusters (e.g. patients clustered within physicians or surgeons) and confounding due to cluster characteristics (e.g. physician or surgeon experience or training). Given the frequency with which these issues arise in medical research, as well as their relative complexity, methods for the analysis of clustered observational data are reviewed. We argue for estimating causal treatment effects using marginal models that either match or weight observations using a suitable distance metric (e.g. the propensity score). Simulation results demonstrated that methods incorporating clustering into calculation of the variance were generally more accurate than those that did not. Moreover, methods that account for cluster confounding when estimating the treatment effect were least biased and most accurate. Throughout the paper we illustrate the proposed methods in a medical device setting that compares the effectiveness of femoral heads used in total hip replacements. Whenever possible the clustered aspect of the data should be considered in the design of the study when constructing the distance measure or in the matching process, as well as in the analysis when estimating the variance of the treatment effect. [ABSTRACT FROM AUTHOR]

One important goal in pharmaco-epidemiology studies is to understand the causal relationship between drug exposures and their clinical outcomes, including adverse drug events. In order to achieve this goal, however, we need to resolve several challenges. Most of pharmaco-epidemiology data are observational and confounding is largely present due to many co-medications. The pharmaco-epidemiology study data set is often sampled from large medical record databases using a matched case-control design, and it may not be representative of the original patient population in the medical record databases. Data analysis method needs to handle a large sample size that cannot be handled using existing statistical analysis packages. In this paper, we tackle these challenges both methodologically and computationally. We propose a conditional causal log-odds ratio (OR) definition to characterize causal effects of drug exposures on a binary adverse drug event adjusting for individual level confounders. Using a case-control design, we present a propensity score estimation using only case samples and we provide sufficient conditions for the consistency of the estimation of the causal log-odds ratio using case-based propensity scores. Computationally, we implement a principle component analysis to reduce high-dimensional confounders. Extensive simulation studies are performed to demonstrate superior performance of our method to existing methods. Finally, we apply the proposed method to analyze drug-induced myopathy data sampled from a de-identified subset of medical record database (close to 5 million patient records), The Indiana Network for Patient Care. Our method identified 70 drug-induced myopathy (p < 0.05) out 72 drugs, which have myoathy side effects on their FDA drug labels. These 70 drugs include three statins who are known for their myopathy side effects. [ABSTRACT FROM AUTHOR]

Many approaches, including traditional parametric modeling and machine learning techniques, have been proposed to estimate propensity scores. This paper describes a new model averaging approach to propensity score estimation in which parametric and nonparametric estimates are combined to achieve covariate balance. Simulation studies are conducted across different scenarios varying in the degree of interactions and nonlinearities in the treatment model. The results show that, based on inverse probability weighting, the proposed propensity score estimator produces less bias and smaller standard errors than existing approaches. They also show that a model averaging approach with the objective of minimizing the average Kolmogorov-Smirnov statistic leads to the best performing IPW estimator. The proposed approach is also applied to a real data set in evaluating the causal effect of formula or mixed feeding versus exclusive breastfeeding on a child's body mass index Z-score at age 4. The data analysis shows that formula or mixed feeding is more likely to lead to obesity at age 4, compared to exclusive breastfeeding. [ABSTRACT FROM AUTHOR]

Individuals differ not only in their background characteristics but also in how they respond to a particular treatment, intervention, or stimulation. In particular, treatment effects may vary systematically by the propensity for treatment. In this paper, we discuss a practical approach to studying heterogeneous treatment effects as a function of the treatment propensity, under the same assumption commonly underlying regression analysis: ignorability. We describe one parametric method and two nonparametric methods for estimating interactions between treatment and the propensity for treatment. For the first method, we begin by estimating propensity scores for the probability of treatment given a set of observed covariates for each unit and construct balanced propensity score strata; we then estimate propensity score stratum-specific average treatment effects and evaluate a trend across them. For the second method, we match control units to treated units based on the propensity score and transform the data into treatment-control comparisons at the most elementary level at which such comparisons can be constructed; we then estimate treatment effects as a function of the propensity score by fitting a nonparametric model as a smoothing device. For the third method, we first estimate nonparametric regressions of the outcome variable as a function of the propensity score separately for treated units and for control units and then take the difference between the two nonparametric regressions. We illustrate the application of these methods with an empirical example of the effects of college attendance on women’s fertility. [ABSTRACT FROM PUBLISHER]

This study explores the usefulness of covariates on equating test scores from nonequivalent test groups. The covariates are captured by an estimated propensity score, which is used as a proxy for latent ability to balance the test groups. The objective is to assess the sensitivity of the equated scores to various misspecifications in the propensity score model. The study assumes a parametric form of the propensity score and evaluates the effects of various misspecification scenarios on equating error. The results, based on both simulated and real testing data, show that (1) omitting an important covariate leads to biased estimates of the equated scores, (2) misspecifying a nonlinear relationship between the covariates and test scores increases the equating standard error in the tails of the score distributions, and (3) the equating estimators are robust against omitting a second-order term as well as using an incorrect link function in the propensity score estimation model. The findings demonstrate that auxiliary information is beneficial for test score equating in complex settings. However, it also sheds light on the challenge of making fair comparisons between nonequivalent test groups in the absence of common items. The study identifies scenarios, where equating performance is acceptable and problematic, provides practical guidelines, and identifies areas for further investigation. [ABSTRACT FROM AUTHOR]

This study investigates the longitudinal and spatial patterns and spillover effects of Low-Income Housing Tax Credit (LIHTC) developments on neighborhood-level income segregation. Focusing on all MSAs in the U.S., the results show that LIHTC units have been spatially clustered in socioeconomically disadvantaged neighborhoods over time. This research also explores the spillover effects of LIHTC units on neighborhood economic status by utilizing propensity scores and weighted linear regression to address a self-selection bias of developers' decisions regarding the location of LIHTC projects. The results suggest that LIHTC developments, in general, are expected to increase the concentration of households that have lower income than the average household income of the MSA. However, in high-poverty neighborhoods, LIHTC developments yield positive spillover effects on neighborhood economic status. Moreover, providing LIHTC units in high-poverty neighborhoods where LIHTC units were built previously in the focal or any adjacent neighborhood also improves neighborhood economic status. [ABSTRACT FROM AUTHOR]

Students who transfer between colleges risk losing credits and decreasing their chances of degree completion. Despite emerging evidence regarding the effectiveness of articulation agreements to address this challenge, it is unclear if these policies support nonlinear transfer pathways—including lateral transfer between 4-year colleges or reverse transfer to 2-year colleges. I use propensity score weighting to examine a statewide articulation agreement in Ohio that established universal credit acceptance for coursework affecting all transfers. Comparing students who completed universally transferrable courses with those who did not, I find no measurable difference in degree attainment among reverse transfers. But there is a positive association with bachelor's degree attainment among lateral transfers, which the findings suggest is related to academic major persistence. [ABSTRACT FROM AUTHOR]

Background: Many studies in psychological and educational research aim to estimate population average treatment effects (PATE) using data from large complex survey samples, and many of these studies use propensity score methods. Recent advances have investigated how to incorporate survey weights with propensity score methods. However, to this point, that work had not been well summarized, and it was not clear how much difference the different PATE estimation methods would make empirically. Purpose: The purpose of this study is to systematically summarize the appropriate use of survey weights in propensity score analysis of complex survey data and use a case study to empirically compare the PATE estimates using multiple analysis methods that include ordinary least squares regression, weighted least squares regression, and various propensity score applications. Methods: We first summarize various propensity score methods that handle survey weights. We then demonstrate the performance of various analysis methods using a nationally representative data set, the Early Childhood Longitudinal Study–Kindergarten to estimate the effects of preschool on children's academic achievement. The correspondence of the results was evaluated using multiple criteria. Results and Conclusions: It is important for researchers to think carefully about their estimand of interest and use methods appropriate for that estimand. If interest is in drawing inferences to the survey target population, it is important to take the survey weights into account, particularly in the outcome analysis stage for estimating the PATE. The case study shows, however, not much difference among various analysis methods in one applied example. [ABSTRACT FROM AUTHOR]

When equating two test forms, the equated scores will be biased if the test groups differ in ability. To adjust for the ability imbalance between nonequivalent groups, a set of common items is often used. When no common items are available, it has been suggested to use covariates correlated with the test scores instead. In this article, we reduce the covariates to a propensity score and equate the test forms with respect to this score. The propensity score is incorporated within the kernel equating framework using poststratification and chained equating. The methods are evaluated using real college admissions test data and through a simulation study. The results show that propensity scores give an increased equating precision in comparison with the equivalent groups design and a smaller mean squared error than by using the covariates directly. Practical implications are also discussed. [ABSTRACT FROM AUTHOR]

We present a randomization-based inferential framework for experiments characterized by a strongly ignorable assignment mechanism where units have independent probabilities of receiving treatment. Previous works on randomization tests often assume these probabilities are equal within blocks of units. We consider the general case where they differ across units and show how to perform randomization tests and obtain point estimates and confidence intervals. Furthermore, we develop rejection-sampling and importance-sampling approaches for conducting randomization-based inference conditional on any statistic of interest, such as the number of treated units or forms of covariate balance. We establish that our randomization tests are valid tests, and through simulation we demonstrate how the rejection-sampling and importance-sampling approaches can yield powerful randomization tests and thus precise inference. Our work also has implications for observational studies, which commonly assume a strongly ignorable assignment mechanism. Most methodologies for observational studies make additional modeling or asymptotic assumptions, while our framework only assumes the strongly ignorable assignment mechanism, and thus can be considered a minimal-assumption approach. [ABSTRACT FROM AUTHOR]

Motivated by a study exploring differences in glycemic control between non-Hispanic black and non-Hispanic white veterans with type 2 diabetes, we aim to address a type of confounding that arises in spatially referenced observational studies. Specifically, we develop a spatial doubly robust propensity score estimator to reduce bias associated with geographic confounding, which occurs when measured or unmeasured confounding factors vary by geographic location, leading to imbalanced group comparisons. We augment the doubly robust estimator with spatial random effects, which are assigned conditionally autoregressive priors to improve inferences by borrowing information across neighboring geographic regions. Through a series of simulations, we show that ignoring spatial variation results in increased absolute bias and mean squared error, while the spatial doubly robust estimator performs well under various levels of spatial heterogeneity and moderate sample sizes. In the motivating application, we construct three global estimates of the risk difference between race groups: an unadjusted estimate, a doubly robust estimate that adjusts only for patient-level information, and a hierarchical spatial doubly robust estimate. Results indicate a gradual reduction in the risk difference at each stage, with the inclusion of spatial random effects providing a 20% reduction compared to an estimate that ignores spatial heterogeneity. Smoothed maps indicate poor glycemic control across Alabama and southern Georgia, areas comprising the so-called "stroke belt." These results suggest the need for community-specific interventions to target diabetes in geographic areas of greatest need. [ABSTRACT FROM AUTHOR]

Propensity score methods are common for estimating a binary treatment effect when treatment assignment is not randomized. When exposure is measured on an ordinal scale (i.e. low-medium-high), however, propensity score inference requires extensions which have received limited attention. Estimands of possible interest with an ordinal exposure are the average treatment effects between each pair of exposure levels. Using these estimands, it is possible to determine an optimal exposure level. Traditional methods, including dichotomization of the exposure or a series of binary propensity score comparisons across exposure pairs, are generally inadequate for identification of optimal levels. We combine subclassification with regression adjustment to estimate transitive, unbiased average causal effects across an ordered exposure, and apply our method on the 2005-2006 National Health and Nutrition Examination Survey to estimate the effects of nutritional label use on body mass index. [ABSTRACT FROM AUTHOR]

Randomized controlled trials (RCTs) and quasi-experimental designs like regression discontinuity (RD) designs, instrumental variable (IV) designs, and matching and propensity score (PS) designs are frequently used for inferring causal effects. It is well known that the features of these designs facilitate the identification of a causal estimand and, thus, warrant a causal interpretation of the estimated effect. In this article, we discuss and compare the identifying assumptions of quasi-experiments using causal graphs. The increasing complexity of the causal graphs as one switches from an RCT to RD, IV, or PS designs reveals that the assumptions become stronger as the researcher’s control over treatment selection diminishes. We introduce limiting graphs for the RD design and conditional graphs for the latent subgroups of compliers, always takers, and never takers of the IV design, and argue that the PS is a collider that offsets confounding bias via collider bias. [ABSTRACT FROM AUTHOR]

This article addresses the issues involved in using compliance with probation supervision as an interim outcome measure in evaluation research. We address the complex nature of compliance and what it implies. Like much research on probation and criminal justice more generally, it was not possible to use random assignment to treatment and comparison groups in the case study we address, which evaluated the SEED training programme. We therefore compare two different data analysis methods to adjust for prior underlying differences between groups, namely regression adjustment of treatment covariates that are related to the outcome measure in the sample data and regression adjustment using propensity scores derived from a wide range of baseline variables. The propensity score method allows for control of a wider range of baseline variables, including those which do not differ significantly between the two groups. [ABSTRACT FROM AUTHOR]

The present study examines the long-term recidivism patterns of a group of unauthorized immigrants identified to be at high risk of recidivism. Using a sample of 517 male unauthorized immigrants, we used three measures of recidivism to assess 9-year rearrest differences between unauthorized immigrants who have and who have not been previously removed from the United States. Results indicate that prior removal was a significant risk marker for recidivism, with previously removed immigrants showing a higher likelihood of rearrest, a greater frequency of rearrest, and a more rapid time-to-first rearrest. While the present study does not establish whether previous removal is a consistent indicator of high recidivism, it suggests that this group of unauthorized immigrants may be worthy of review and policy consideration. Much potential value for law enforcement lies in the sharing of federal immigration records with academics to further study the outcomes of unauthorized immigrants. [ABSTRACT FROM AUTHOR]

We implement a propensity score matching technique to present the first evidence on the impact of professional sports lockouts on player productivity. In particular, we utilize a unique natural experiment from the 2012-2013 National Hockey League lockout, during which approximately 200 players decided to play overseas, while the rest stayed in North America. We separate players based on their nationality and investigate the effect of playing abroad on postlockout player performance. We find limited evidence of enhanced productivity among European players and no evidence of a benefit or drawback for North American players. Our study contributes to the understanding of lockouts in professional sports and the general discussion of labor disputes and worker productivity. [ABSTRACT FROM AUTHOR]

As policymakers contemplate expanding preschool opportunities for low-income children, one possibility is to fund 2, rather than 1 year of Head Start for children at ages 3 and 4. Another option is to offer 1 year of Head Start followed by 1 year of pre-K. We ask which of these options is more effective. We use data from the Oklahoma pre-K study to examine these two “pathways” into kindergarten using regression discontinuity to estimate the effects of each age 4 program, and propensity score weighting to address selection. We find that children attending Head Start at age 3 develop stronger prereading skills in a high-quality pre-kindergarten at age 4 compared with attending Head Start at age 4. Pre-K and Head Start were not differentially linked to improvements in children’s prewriting skills or premath skills. This suggests that some impacts of early learning programs may be related to the sequencing of learning experiences to more academic programming. [ABSTRACT FROM AUTHOR]

The framework of principal stratification provides a way to think about treatment effects conditional on post-randomization variables, such as level of compliance. In particular, the complier average causal effect (CACE) - the effect of the treatment for those individuals who would comply with their treatment assignment under either treatment condition - is often of substantive interest. However, estimation of the CACE is not always straightforward, with a variety of estimation procedures and underlying assumptions, but little advice to help researchers select between methods. In this article, we discuss and examine two methods that rely on very different assumptions to estimate the CACE: a maximum likelihood ('joint') method that assumes the 'exclusion restriction,' (ER) and a propensity score-based method that relies on 'principal ignorability.' We detail the assumptions underlying each approach, and assess each methods' sensitivity to both its own assumptions and those of the other method using both simulated data and a motivating example. We find that the ER-based joint approach appears somewhat less sensitive to its assumptions, and that the performance of both methods is significantly improved when there are strong predictors of compliance. Interestingly, we also find that each method performs particularly well when the assumptions of the other approach are violated. These results highlight the importance of carefully selecting an estimation procedure whose assumptions are likely to be satisfied in practice and of having strong predictors of principal stratum membership. [ABSTRACT FROM AUTHOR]

Background: Police agencies around the country are implementing various strategies to reduce crime in their communities that need to be evaluated. These strategies are often based on systematic crime analysis and are focused on crime occurring in hot spots, which are areas of disproportionate amounts of crime. Objective: This article takes a practice-based research approach to evaluate whether evidence-based police strategies implemented by one police agency as its normal everyday crime reduction practice are effective in reducing residential burglary incidents in micro-time hot spots. Research Design: A quasi-experimental ex post facto design is employed using 5 years of data from one police agency that has institutionalized the identification and response to micro-time hot spots into its day-to-day practices. Propensity score matching is used to match 54 pairs of micro-time hot spots using logistic regression to compute the propensity scores and greedy 1 to 1 matching with a caliper width of 0.5 of the standard deviation of the logit to match the cases. Results: Independent t-tests show that tactical police response to micro-time hot spots can lead to significant reductions in residential burglary incidents without the spatial displacement of crime. Conclusion: Tactical police responses that seek to achieve short-term reductions in crime appear to be well suited for micro-time hot spots since they are, by nature, short term. Importantly, the conclusions are based on the evaluation of an agency’s systematic implementation of the evidence-based practices as its normal practices and not for the sake of research. [ABSTRACT FROM AUTHOR]

Football hooliganism has long occupied the social, media, and political landscapes of fans around the world. In this study, we use data from the Cambridge Study in Delinquent Development, a longitudinal study of a large sample of males followed from childhood to late middle adulthood, to examine the relationship between involvement in football hooliganism and both conviction frequency and conviction trajectories. Results show that, although hooligans differ from non-hooligans on several childhood and teenage risk factors, and that they also incur many more convictions than their non-hooligan counterparts, these relationships do not hold after carefully matching hooligans and non-hooligans on key risk factors. Football hooliganism, for the most part, seems to be part of a constellation of antisocial behaviors that also includes criminal offending. [ABSTRACT FROM AUTHOR]

Increasing nonresponse rates in federal surveys and potentially biased survey estimates are a growing concern, especially with regard to establishment surveys. Unlike household surveys, not all establishments contribute equally to survey estimates. With regard to agricultural surveys, if an extremely large farm fails to complete a survey, the United States Department of Agriculture (USDA) could potentially underestimate average acres operated among other things. In order to identify likely nonrespondents prior to data collection, the USDA's National Agricultural Statistics Service (NASS) began modeling nonresponse using Census of Agriculture data and prior Agricultural Resource Management Survey (ARMS) response history. Using an ensemble of classification trees, NASS has estimated nonresponse propensities for ARMS that can be used to predict nonresponse and are correlated with key ARMS estimates. [ABSTRACT FROM AUTHOR]

Background: A small but growing body of research evidence suggests that place-based police interventions generate significant crime control gains. While place-based policing strategies have been adopted by a majority of U.S. police departments, very few agencies make a priori commitments to rigorous evaluations. Objective: Recent methodological developments were applied to conduct a rigorous ex post facto evaluation of the Boston Police Department’s Safe Street Team (SST) hot spots policing program. Research Design: A nonrandomized quasi-experimental design was used to evaluate the violent crime control benefits of the SST program at treated street segments and intersections relative to untreated street segments and intersections. Propensity score matching techniques were used to identify comparison places in Boston. Growth curve regression models were used to analyze violent crime trends at treatment places relative to control places. Units of Analysis: Using computerized mapping and database software, a micro-level place database of violent index crimes at all street segments and intersections in Boston was created. Measures: Yearly counts of violent index crimes between 2000 and 2009 at the treatment and comparison street segments and intersections served as the key outcome measure. Results: The SST program was associated with a statistically significant reduction in violent index crimes at the treatment places relative to the comparison places without displacing crime into proximate areas. Conclusions: To overcome the challenges of evaluation in real-world settings, evaluators need to continuously develop innovative approaches that take advantage of new theoretical and methodological approaches. [ABSTRACT FROM PUBLISHER]

Previously deported aliens are a group about which numerous claims are made but very few facts are known. Using data on male deportable aliens released from a local jail, the study sought to test the ubiquitous claim that they pose a high risk of recidivism. Using multiple measures of recidivism and propensity score weighting to account for preexisting group differences, the authors find consistent support for the assertion that previously deported aliens are a high recidivism risk. Relative to similarly situated deportable aliens with no record of deportation, previously deported aliens are more likely to be rearrested, to be rearrested more quickly, and to be rearrested more frequently in a 1-year follow-up period. [ABSTRACT FROM AUTHOR]

Adjustment methods such as propensity scores and analysis of covariance are often used for estimating treatment effects in nonexperimental data. Shadish, Clark, and Steiner used a within-study comparison to test how well these adjustments work in practice. They randomly assigned participating students to a randomized or nonrandomized experiment. Treatment effects were then estimated in the experiment and compared to the adjusted nonexperimental estimates. Most of the selection bias in the nonexperiment was reduced. The present study replicates the study of Shadish et al. despite some differences in design and in the size and direction of the initial bias. The results show that the selection of covariates matters considerably for bias reduction in nonexperiments but that the choice of analysis matters less. [ABSTRACT FROM AUTHOR]

Findings from an evaluation of a model system for delivering mental health services to youth were reassessed to determine the robustness of key findings to the use of methodologies unavailable to the original analysts. These analyses address a key concern about earlier findings—that the quasi-experimental design involved the comparison of two noncomparable groups. The authors employed propensity score methodology to reconsider between-group baseline differences in observed characteristics of participating families. The authors also considered the possible effect of unobserved between-group differences. The data support previous studies that show few differences in outcomes, but the findings are sensitive to unobserved heterogeneity. [ABSTRACT FROM AUTHOR]

Web surveys are a popular survey mode, but the subpopulation with Internet access may not represent the population of interest. The authors investigate whether adjusting using weights or matching on a small set of variables makes the distributions of target variables representative of the population. This application has a rich sampling design; the Internet sample is part of an existing probability sample, the Health and Retirement Study, that is representative of the U.S. population aged 50 and older. For the dichotomous variables investigated, the adjustment helps. On average, the sample means in the Internet access sample differ by 6.5 percent before and 3.7 percent after adjustment. Still, a large number of adjusted estimates remain significantly different from their target estimates based on the complete sample. This casts doubt on the common procedure to use only a few variables to correct for the selectivity of convenience samples. [ABSTRACT FROM AUTHOR]

Regressions can be weighted by propensity scores in order to reduce bias. However, weighting is likely to increase random error in the estimates, and to bias the estimated standard errors downward, even when selection mechanisms are well understood. Moreover, in some cases, weighting will increase the bias in estimated causal parameters. If investigators have a good causal model, it seems better just to fit the model without weights. If the causal model is improperly specified, there can be significant problems in retrieving the situation by weighting, although weighting may help under some circumstances. [ABSTRACT FROM AUTHOR]