Search

Maryse C Cnossen,1 Thomas A van Essen,2,3 Iris E Ceyisakar,1 Suzanne Polinder,1 Teuntje M Andriessen,4 Joukje van der Naalt,5 Iain Haitsma,6 Janneke Horn,7 Gaby Franschman,8 Pieter E Vos,9 Wilco C Peul,2,3 David K Menon,10 Andrew IR Maas,11 Ewout W Steyerberg,1,12 Hester F Lingsma1 1Center for Medical Decision Making, Department of Public Health, Erasmus Medical Center Rotterdam, Rotterdam, the Netherlands; 2Neurosurgical Cooperative Holland, Department of Neurosurgery, Leiden University Medical Center, Leiden, the Netherlands; 3Department of Neurosurgery, Haaglanden Medical Center, The Hague, the Netherlands; 4Department of Psychology, RIVAS Healthcare Group, Gorinchem, the Netherlands; 5Department of Neurology, University Medical Center Groningen, Groningen, the Netherlands; 6Department of Neurosurgery, Erasmus Medical Center Rotterdam, Rotterdam, the Netherlands; 7Department of Intensive Care Medicine, Academic Medical Center, University of Amsterdam, Amsterdam, the Netherlands; 8Department of Anesthesiology, VU University Medical Center Amsterdam, Amsterdam, the Netherlands; 9Department of Neurology, Slingeland Hospital, Doetinchem, the Netherlands; 10Division of Anaesthesia, University of Cambridge/Addenbrooke’s Hospital, Cambridge, UK; 11Department of Neurosurgery, Antwerp University Hospital and University of Antwerp, Edegem, Belgium; 12Department of Medical Statistics and Bioinformatics, Leiden University Medical Center, Leiden, the Netherlands Introduction: Observational studies of interventions are at risk for confounding by indication. The objective of the current study was to define the circumstances for the validity of methods to adjust for confounding by indication in observational studies. Patients and methods: We performed post hoc analyses of data prospectively collected from three European and North American traumatic brain injury studies including 1,725 patients. The effects of three interventions (intracranial pressure [ICP] monitoring, intracranial operation and primary referral) were estimated in a proportional odds regression model with the Glasgow Outcome Scale as ordinal outcome variable. Three analytical methods were compared: classical covariate adjustment, propensity score matching and instrumental variable (IV) analysis in which the percentage exposed to an intervention in each hospital was added as an independent variable, together with a random intercept for each hospital. In addition, a simulation study was performed in which the effect of a hypothetical beneficial intervention (OR 1.65) was simulated for scenarios with and without unmeasured confounders. Results: For all three interventions, covariate adjustment and propensity score matching resulted in negative estimates of the treatment effect (OR ranging from 0.80 to 0.92), whereas the IV approach indicated that both ICP monitoring and intracranial operation might be beneficial (OR per 10% change 1.17, 95% CI 1.01–1.42 and 1.42, 95% CI 0.95–1.97). In our simulation study, we found that covariate adjustment and propensity score matching resulted in an invalid estimate of the treatment effect in case of unmeasured confounders (OR ranging from 0.90 to 1.03). The IV approach provided an estimate in the similar direction as the simulated effect (OR per 10% change 1.04–1.05) but was statistically inefficient. Conclusion: The effect estimation of interventions in observational studies strongly depends on the analytical method used. When unobserved confounding and practice variation are expected in observational multicenter studies, IV analysis should be considered. Keywords: confounding, observational studies, traumatic brain injury, instrumental variable analysis, comparative effectiveness research

Background: In traumatic brain injury (TBI), large between-center differences in treatment and outcome for patients managed in the intensive care unit (ICU) have been shown. The aim of this study is to explore if European neurotrauma centers can be clustered, based on their treatment preference in different domains of TBI care in the ICU., Methods: Provider profiles of centers participating in the Collaborative European Neurotrauma Effectiveness Research in TBI study were used to assess correlations within and between the predefined domains: intracranial pressure monitoring, coagulation and transfusion, surgery, prophylactic antibiotics, and more general ICU treatment policies. Hierarchical clustering using Ward's minimum variance method was applied to group data with the highest similarity. Heat maps were used to visualize whether hospitals could be grouped to uncover types of hospitals adhering to certain treatment strategies., Results: Provider profiles were available from 66 centers in 20 different countries in Europe and Israel. Correlations within most of the predefined domains varied from low to high correlations (mean correlation coefficients 0.2-0.7). Correlations between domains were lower, with mean correlation coefficients of 0.2. Cluster analysis showed that policies could be grouped, but hospitals could not be grouped based on their preference., Conclusions: Although correlations between treatment policies within domains were found, the failure to cluster hospitals indicates that a specific treatment choice within a domain is not a proxy for other treatment choices within or outside the domain. These results imply that studying the effects of specific TBI interventions on outcome can be based on between-center variation without being substantially confounded by other treatments., Trial Registration: We do not report the results of a health care intervention., (© 2021. The Author(s).)

Background: Instrumental variable (IV) analysis holds the potential to estimate treatment effects from observational data. IV analysis potentially circumvents unmeasured confounding but makes a number of assumptions, such as that the IV shares no common cause with the outcome. When using treatment preference as an instrument, a common cause, such as a preference regarding related treatments, may exist. We aimed to explore the validity and precision of a variant of IV analysis where we additionally adjust for the provider: adjusted IV analysis., Methods: A treatment effect on an ordinal outcome was simulated (beta - 0.5 in logistic regression) for 15.000 patients, based on a large data set (the IMPACT data, n = 8799) using different scenarios including measured and unmeasured confounders, and a common cause of IV and outcome. We compared estimated treatment effects with patient-level adjustment for confounders, IV with treatment preference as the instrument, and adjusted IV, with hospital added as a fixed effect in the regression models., Results: The use of patient-level adjustment resulted in biased estimates for all the analyses that included unmeasured confounders, IV analysis was less confounded, but also less reliable. With correlation between treatment preference and hospital characteristics (a common cause) estimates were skewed for regular IV analysis, but not for adjusted IV analysis., Conclusion: When using IV analysis for comparing hospitals, some limitations of regular IV analysis can be overcome by adjusting for a common cause., Trial Registration: We do not report the results of a health care intervention., (© 2022. The Author(s).)

Background: There is a growing interest in assessment of the quality of hospital care, based on outcome measures. Many quality of care comparisons rely on binary outcomes, for example mortality rates. Due to low numbers, the observed differences in outcome are partly subject to chance. We aimed to quantify the gain in efficiency by ordinal instead of binary outcome analyses for hospital comparisons. We analyzed patients with traumatic brain injury (TBI) and stroke as examples., Methods: We sampled patients from two trials. We simulated ordinal and dichotomous outcomes based on the modified Rankin Scale (stroke) and Glasgow Outcome Scale (TBI) in scenarios with and without true differences between hospitals in outcome. The potential efficiency gain of ordinal outcomes, analyzed with ordinal logistic regression, compared to dichotomous outcomes, analyzed with binary logistic regression was expressed as the possible reduction in sample size while keeping the same statistical power to detect outliers., Results: In the IMPACT study (9578 patients in 265 hospitals, mean number of patients per hospital = 36), the analysis of the ordinal scale rather than the dichotomized scale ('unfavorable outcome'), allowed for up to 32% less patients in the analysis without a loss of power. In the PRACTISE trial (1657 patients in 12 hospitals, mean number of patients per hospital = 138), ordinal analysis allowed for 13% less patients. Compared to mortality, ordinal outcome analyses allowed for up to 37 to 63% less patients., Conclusions: Ordinal analyses provide the statistical power of substantially larger studies which have been analyzed with dichotomization of endpoints. We advise to exploit ordinal outcome measures for hospital comparisons, in order to increase efficiency in quality of care measurements., Trial Registration: We do not report the results of a health care intervention.

Background: Comparing outcomes across hospitals to learn from best performing hospitals can be valuable. However, reliably identifying best performance is challenging. This study assesses the possibility to distinguish best performing hospitals on single outcomes and consistency of performance on different outcomes., Methods: Data were derived from the Dutch ColoRectal Audit 2013-2015. Outcomes considered were textbook outcome (colon), (circumferential) resection margins, (serious) complications, mortality, and 'failure to rescue'. To include uncertainty in rankings, random effect logistic regression models were used to calculate expected ranks (ERs), for each hospital and outcome. Rankability was calculated for each outcome, as a measure of reliability of ranking. Furthermore, correlation between ERs on different outcomes was assessed. Correlation was considered weak <0.40, moderate between 0.40 - 0.59 and strong >0.60., Results: The study included 32 143 patients; of whom 11 373 were treated in 2015 across 84 hospitals, 8181 colon and 3192 rectal cancer patients. In this one-year period 'Postoperative complications' had the highest rankability for colon (57%) and rectal (41%) surgery. No (group of) hospital(s) had the highest ER(s) on all outcomes. Correlation between ERs of outcomes was moderate in 2 (of 25) and strong in 4 (of 25) combinations. Rankability of colorectal mortality increased from 14% in 2015 to 35% when data over 2013-2015 were used., Conclusion: The highest reliability of identifying best performance based on an outcome was 57%. However, the balance between reliability and relevance of outcomes is vulnerable. No (group of) hospital(s) could be identified as best performer on all outcomes. Performance was not consistent on outcomes., Competing Interests: Declaration of competing interest None of the authors have anything to disclose., (Copyright © 2020. Published by Elsevier Ltd.)

Background: Report cards on the health care system increasingly report provider-specific performance on indicators that measure the quality of health care delivered. A natural reaction to the publishing of hospital-specific performance on a given indicator is to create 'league tables' that rank hospitals according to their performance. However, many indicators have been shown to have low to moderate rankability, meaning that they cannot be used to accurately rank hospitals. Our objective was to define conditions for improving the ability to rank hospitals by combining several binary indicators with low to moderate rankability., Methods: Monte Carlo simulations to examine the rankability of composite ordinal indicators created by pooling three binary indicators with low to moderate rankability. We considered scenarios in which the prevalences of the three binary indicators were 0.05, 0.10, and 0.25 and the within-hospital correlation between these indicators varied between - 0.25 and 0.90., Results: Creation of an ordinal indicator with high rankability was possible when the three component binary indicators were strongly correlated with one another (the within-hospital correlation in indicators was at least 0.5). When the binary indicators were independent or weakly correlated with one another (the within-hospital correlation in indicators was less than 0.5), the rankability of the composite ordinal indicator was often less than at least one of its binary components. The rankability of the composite indicator was most affected by the rankability of the most prevalent indicator and the magnitude of the within-hospital correlation between the indicators., Conclusions: Pooling highly-correlated binary indicators can result in a composite ordinal indicator with high rankability. Otherwise, the composite ordinal indicator may have lower rankability than some of its constituent components. It is recommended that binary indicators be combined to increase rankability only if they represent the same concept of quality of care.

Background: Despite widespread use of quality indicators, it remains unclear to what extent they can reliably distinguish hospitals on true differences in performance. Rankability measures what part of variation in performance reflects 'true' hospital differences in outcomes versus random noise., Objective: This study sought to assess whether combining data into composites or including data from multiple years improves the reliability of ranking quality indicators for hospital care., Methods: Using the Dutch National Medical Registration (2007-2012) for stroke, colorectal carcinoma, heart failure, acute myocardial infarction and total hiparthroplasty (THA)/ total knee arthroplasty (TKA) in osteoarthritis (OA), we calculated the rankability for in-hospital mortality, 30-day acute readmission and prolonged length of stay (LOS) for single years and 3-year periods and for a dichotomous and ordinal composite measure in which mortality, readmission and prolonged LOS were combined. Rankability, defined as (between-hospital variation/between-hospital+within hospital variation)×100% is classified as low (<50%), moderate (50%-75%) and high (>75%)., Results: Admissions from 555 053 patients treated in 95 hospitals were included. The rankability for mortality was generally low or moderate, varying from less than 1% for patients with OA undergoing THA/TKA in 2011 to 71% for stroke in 2010. Rankability for acute readmission was low, except for acute myocardial infarction in 2009 (51%) and 2012 (62%). Rankability for prolonged LOS was at least moderate. Combining multiple years improved rankability but still remained low in eight cases for both mortality and acute readmission. Combining the individual indicators into the dichotomous composite, all diagnoses had at least moderate rankability (range: 51%-96%). For the ordinal composite, only heart failure had low rankability (46% in 2008) (range: 46%-95%)., Conclusion: Combining multiple years or into multiple indicators results in more reliable ranking of hospitals, particularly compared with mortality and acute readmission in single years, thereby improving the ability to detect true hospital differences. The composite measures provide more information and more reliable rankings than combining multiple years of individual indicators., Competing Interests: Competing interests: None declared., (© Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2019. All rights reserved. No commercial use is permitted unless otherwise expressly granted.)

Background: For many major surgical procedures, the outcomes are better when performed by surgeons with higher procedure volumes. The purpose of this study was to examine the relations between surgeon procedure volume and the outcomes of Dupuytren's surgery., Methods: The authors conducted an observational study from 2011 to 2014 at six dedicated hand surgery practice sites in The Netherlands. Five hundred eighty-eight patients underwent surgery for Dupuytren's contracture performed by one of the 16 surgeons. The main exposure variable was annual surgeon volume. Outcome measures were the degree of residual contracture, full release rate, and any postoperative adverse event examined within 3 months of surgery., Results: Mean annual surgeon volume was 51 among the 16 surgeons(range; 4-86) procedures. The majority of patients had primary disease (79 percent) and underwent open fasciectomy (74 percent). Multivariable regression analyses showed that surgeon volume was linearly related to all three outcomes, and identified no optimal volume threshold. Performing 10 additional procedures annually was independently associated with nearly 0.8 degree less residual contracture (p = 0.002), 9 percent higher odds of attaining a full release (p = 0.037), and 11 percent lower odds of an adverse event (p < 0.001). Nonetheless, patient-related factors had larger impacts on all three clinical outcomes than surgeon volume., Conclusions: In this study of practicing hand surgeons, surgeon volume varied widely, and a higher volume was associated with less postoperative residual contracture, higher full release rates, and fewer adverse events. This implies that increasing surgeon's procedure volume provides an opportunity for improving the outcomes of Dupuytren's surgery., Clinical Question/level of Evidence: Therapeutic, III.