Challenges of an attempted cross-national comparison of healthcare and social care utilization and costs in patients with congestive heart failure in the United States and Netherlands

Introduction: We embarked on a research project intending to compare healthcare and social care utilization and costs for people with congestive heart failure in the United States (US) and the Netherlands (NL). Our intention was to evaluate these characteristics at a national, regional, and person-specific level, comparing data from patients being cared for at Hennepin Healthcare (Minneapolis, MN, US) and Radboud University Medical Center (Nijmegen, NL). Because we focused on a specific clinical subpopulation in both countries, we expected to be able to make valid comparisons between these two patient populations. However, our experience with this project calls into question the accuracy and reliability of between-country healthcare comparisons. Methods: Spurious and unexpected results from the between-country database comparison prompted us to critically examine our findings. We used iterative group discussions to identify weaknesses and missing components in our comparative data. In addition to exploring specific examples from our own unpublished work, we went on to identify similar limitations in external, peer-reviewed publications. Results: We identified five domains that limited our ability to perform a valid between-country comparison. These included (1) differences in patient level characteristics both within and between countries; (2) variations in societal norms and values; (3) different systems of healthcare and social care organization and delivery; (4) variable definitions, implantation, and distribution of costs; and (5) limitations within the databases themselves. Discussion: Iterative group discussion and reflection were useful in understanding defining limitations in our between country database comparison. Acceptance of the database comparison would have led to erroneous conclusions about heart failure patients in the US and NL. Conclusions: There are five domains that limited our ability to perform a valid between-country comparison, including patient-level characteristics, societal norms and values, systems of healthcare and social care, costs, and databases. Lessons Learned: Our findings suggest that these domains should be considered prior to undertaking an international comparison of healthcare and social care. They may also provide a tool for critical analysis of published data. Limitations: The five domains we identified are based on a single study. However, we were able to identify external examples in peer-reviewed journals in each of the domains. Suggestions for future research: In order for us to continue to learn from other systems, international comparisons of healthcare and social care should continue, but particular attention must be given to ensuring that accurate and reliable comparisons are made. The domains we have identified may stimulate further research into quality assurance for international healthcare comparisons.