Matching on propensity and prognostic scores can lead to different estimates of heterogeneous treatment effects: a case study and simulation.

The purpose of this study is to illustrate how matching approaches based on different balancing scores lead to variations in the treatment effect estimators. We introduced a case study evaluating the effect of anti-thrombotic agents on the severe progression of patients with intracerebral hemorrhage. We extracted subpopulations based on propensity and prognostic scores and then estimated the relative risk between the treatment groups. Furthermore, to illustrate the situation where the treatment effect estimates varied depending on employed balancing scores, we conducted a simulation experiment. In the case study, the matching using different balancing scores extracted subpopulations with different characteristics. Then, the estimated relative risk (95% confidence interval) was 1.27 (0.98–1.94) among the propensity score matched cohort, whereas it was 0.91 (0.76–1.08) among the prognostic score matched cohort. In the simulation experiments, the results indicated that the matching schemes based on different balancing scores created distinct matched cohorts, leading to varying estimates under treatment effect heterogeneity. Moreover, the variability of the estimated effect becomes substantial when there are covariates strongly related to the dependent variable of the nuisance functions. The difference in the selected subpopulation via matching based on different balancing scores is a thoughtful factor that can result in different estimates when there is effect heterogeneity. In practice, we recommend assessing the characteristics of the matched subpopulation and employing the balancing score that can estimate the treatment effect among the target population of interest in each study. Plain Language Summary: This study aimed to understand why different methods of matching patients in a study can lead to different estimates of how well a treatment works. We used a real example of a study where the treatment's effectiveness varied for different patients and compared how different methods of matching patients affected the results. We also did a series of experiments to see how the different methods affected estimates of treatment effect when the effectiveness varied among patients. Our study found that the different matching methods extracted the different patients. The difference in the matched patients led to very different results, especially when the treatment's effectiveness was not the same for all patients. Therefore, it is important to check the characteristics of patients after matching and consider whether the treatment's effectiveness is not the same for all patients. [ABSTRACT FROM AUTHOR]