Search

Missing data are frequently encountered in registries that are used to compare performance across hospitals. The most appropriate method for handling missing data when analysing differences in outcomes between hospitals with a generalised linear mixed model is unclear. We aimed to compare methods for handling missing data when comparing hospitals on ordinal and dichotomous outcomes. We performed a simulation study using data from the Multicentre Randomised Controlled Trial of Endovascular Treatment for Acute Ischaemic Stroke in the Netherlands (MR CLEAN) Registry, a prospective cohort study in 17 hospitals performing endovascular therapy for ischaemic stroke in the Netherlands. The investigated methods for handling missing data, both case-mix adjustment variables and outcomes, were complete case analysis, single imputation, multiple imputation, single imputation with deletion of imputed outcomes and multiple imputation with deletion of imputed outcomes. Data were generated as missing completely at random (MCAR), missing at random and missing not at random (MNAR) in three scenarios: (1) 10% missing data in case-mix and outcome; (2) 40% missing data in case-mix and outcome; and (3) 40% missing data in case-mix and outcome with varying degree of missing data among hospitals. Bias and reliability of the methods were compared on the mean squared error (MSE, a summary measure combining bias and reliability) relative to the hospital effect estimates from the complete reference data set. For both the ordinal outcome (ie, the modified Rankin Scale) and a common dichotomised version thereof, all methods of handling missing data were biased, likely due to shrinkage of the random effects. The MSE of all methods was on average lowest under MCAR and with fewer missing data, and highest with more missing data and under MNAR. The 'multiple imputation, then deletion' method had the lowest MSE for both outcomes under all simulated patterns of missing data. Thus, when estimating hospital effects on ordinal and dichotomous outcomes in the presence of missing data, the least biased and most reliable method to handle these missing data is 'multiple imputation, then deletion'., Competing Interests: Competing interests: WvZ has received consulting fees from Codman and Stryker. DD has received research grants from Stryker and Bracco Imaging., (© Author(s) (or their employer(s)) 2023. No commercial re-use. See rights and permissions. Published by BMJ.)

Background Insight into outcome variation between hospitals could help to improve quality of care. We aimed to assess the validity of early outcomes as quality indicators for acute ischemic stroke care for patients treated with endovascular therapy (EVT). Methods and Results We used data from the MR CLEAN (Multicenter Randomized Controlled Trial of Endovascular Treatment for Acute Ischemic Stroke in the Netherlands) Registry, a large multicenter prospective cohort study including 3279 patients with acute ischemic stroke undergoing EVT. Random effect linear and proportional odds regression were used to analyze the effect of case mix on between-hospital differences in 2 early outcomes: the National Institutes of Health Stroke Scale (NIHSS) score at 24 to 48 hours and the expanded thrombolysis in cerebral infarction score. Between-hospital variation in outcomes was assessed using the variance of random hospital effects (tau 2 ). In addition, we estimated the correlation between hospitals' EVT-patient volume and (case-mix-adjusted) outcomes. Both early outcomes and case-mix characteristics varied significantly across hospitals. Between-hospital variation in the expanded thrombolysis in cerebral infarction score was not influenced by case-mix adjustment (tau 2 =0.17 in both models). In contrast, for the NIHSS score at 24 to 48 hours, case-mix adjustment led to a decrease in variation between hospitals (tau 2 decreases from 0.19 to 0.17). Hospitals' EVT-patient volume was strongly correlated with higher expanded thrombolysis in cerebral infarction scores ( r =0.48) and weakly with lower NIHSS score at 24 to 48 hours ( r =0.15). Conclusions Between-hospital variation in NIHSS score at 24 to 48 hours is significantly influenced by case-mix but not by patient volume. In contrast, between-hospital variation in expanded thrombolysis in cerebral infarction score is strongly influenced by EVT-patient volume but not by case-mix. Both outcomes may be suitable for comparing hospitals on quality of care, provided that adequate adjustment for case-mix is applied for NIHSS score.

Introduction: Various statistical approaches can be used to deal with unmeasured confounding when estimating treatment effects in observational studies, each with its own pros and cons. This study aimed to compare treatment effects as estimated by different statistical approaches for two interventions in observational stroke care data., Patients and Methods: We used prospectively collected data from the MR CLEAN registry including all patients (n = 3279) with ischemic stroke who underwent endovascular treatment (EVT) from 2014 to 2017 in 17 Dutch hospitals. Treatment effects of two interventions - i.e., receiving an intravenous thrombolytic (IVT) and undergoing general anesthesia (GA) before EVT - on good functional outcome (modified Rankin Scale ≤2) were estimated. We used three statistical regression-based approaches that vary in assumptions regarding the source of unmeasured confounding: individual-level (two subtypes), ecological, and instrumental variable analyses. In the latter, the preference for using the interventions in each hospital was used as an instrument., Results: Use of IVT (range 66-87%) and GA (range 0-93%) varied substantially between hospitals. For IVT, the individual-level (OR ~ 1.33) resulted in significant positive effect estimates whereas in instrumental variable analysis no significant treatment effect was found (OR 1.11; 95% CI 0.58-1.56). The ecological analysis indicated no statistically significant different likelihood (β = - 0.002%; P = 0.99) of good functional outcome at hospitals using IVT 1% more frequently. For GA, we found non-significant opposite directions of points estimates the treatment effect in the individual-level (ORs ~ 0.60) versus the instrumental variable approach (OR = 1.04). The ecological analysis also resulted in a non-significant negative association (0.03% lower probability)., Discussion and Conclusion: Both magnitude and direction of the estimated treatment effects for both interventions depend strongly on the statistical approach and thus on the source of (unmeasured) confounding. These issues should be understood concerning the specific characteristics of data, before applying an approach and interpreting the results. Instrumental variable analysis might be considered when unobserved confounding and practice variation is expected in observational multicenter studies., (© 2022. The Author(s).)

Background: Although the provision of performance feedback to healthcare professionals based on data from quality registries is common practice in many fields of medicine, observational studies of its effect on the quality of care have shown mixed results. The objective of this study is to evaluate the effect of performance feedback on the quality of care for acute ischemic stroke., Methods: PERFEQTOS is a stepped wedge cluster randomized trial in 13 hospitals in the Netherlands providing endovascular thrombectomy for ischemic stroke. The primary outcome is the hospital's door-to-groin time. The study starts with a 6-month period in which none of the hospitals receives the performance feedback intervention. Subsequently, every 6 months, three or four hospitals are randomized to cross over from the control to the intervention conditions, until all hospitals receive the feedback intervention. The feedback intervention consists of a dashboard with quarterly reports on patient characteristics, structure, process, and outcome indicators related to patients with ischemic stroke treated with endovascular thrombectomy. Hospitals can compare their present performance with their own performance in the past and with other hospitals. The performance feedback is provided to local quality improvement teams in each hospital, who define their own targets on specific indicators and develop performance improvement plans. The impact of the performance feedback and improvement plans will be evaluated by comparing the primary outcome before and after the intervention., Discussion: This study will provide evidence on the effectiveness of performance feedback to healthcare providers. The results will be actively disseminated through peer-reviewed journals, conference presentations, and various stakeholder engagement activities., Trial Registration: Netherlands Trial Register NL9090 . Registered on December 3, 2020., (© 2021. The Author(s).)

Background: Between-center variation in outcome may offer opportunities to identify variation in quality of care. By intervening on these quality differences, patient outcomes may be improved. However, whether observed differences in outcome reflect the true quality improvement potential is not known for many diseases. Therefore, we aimed to analyze the effect of differences in performance on structure and processes of care, and case-mix on between-center differences in outcome after endovascular treatment (EVT) for ischemic stroke., Methods: In this observational cohort study, ischemic stroke patients who received EVT between 2014 and 2017 in all 17 Dutch EVT-centers were included. Primary outcome was the modified Rankin Scale, ranging from 0 (no symptoms) to 6 (death), at 90 days. We used random effect proportional odds regression modelling, to analyze the effect of differences in structure indicators (center volume and year of admission), process indicators (time to treatment and use of general anesthesia) and case-mix, by tracking changes in tau 2 , which represents the amount of between-center variation in outcome., Results: Three thousand two hundred seventy-nine patients were included. Performance on structure and process indicators varied significantly between EVT-centers (P < 0.001). Predicted probability of good functional outcome (modified Rankin Scale 0-2 at 90 days), which can be interpreted as an overall measure of a center's case-mix, varied significantly between 17 and 50% across centers. The amount of between-center variation (tau 2 ) was estimated at 0.040 in a model only accounting for random variation. This estimate more than doubled after adding case-mix variables (tau 2 : 0.086) to the model, while a small amount of between-center variation was explained by variation in performance on structure and process indicators (tau 2 : 0.081 and 0.089, respectively). This indicates that variation in case-mix affects the differences in outcome to a much larger extent., Conclusions: Between-center variation in outcome of ischemic stroke patients mostly reflects differences in case-mix, rather than differences in structure or process of care. Since the latter two capture the real quality improvement potential, these should be used as indicators for comparing center performance. Especially when a strong association exists between those indicators and outcome, as is the case for time to treatment in ischemic stroke.

Background: Patient-Reported Outcome Measures (PROMs) have been proposed for benchmarking health care quality across hospitals, which requires extensive case-mix adjustment. The current study's aim was to develop and compare case-mix models for mortality, a functional outcome, and a patient-reported outcome measure (PROM) in ischemic stroke care., Methods: Data from ischemic stroke patients, admitted to four stroke centers in the Netherlands between 2014 and 2016 with available outcome information (N = 1022), was analyzed. Case-mix adjustment models were developed for mortality, modified Rankin Scale (mRS) scores and EQ-5D index scores with respectively binary logistic, proportional odds and linear regression models with stepwise backward selection. Predictive ability of these models was determined with R-squared (R 2 ) and area-under-the-receiver-operating-characteristic-curve (AUC) statistics., Results: Age, NIHSS score on admission, and heart failure were the only common predictors across all three case-mix adjustment models. Specific predictors for the EQ-5D index score were sex (β = 0.041), socio-economic status (β = - 0.019) and nationality (β = - 0.074). R 2 -values for the regression models for mortality (5 predictors), mRS score (9 predictors) and EQ-5D utility score (12 predictors), were respectively R 2  = 0.44, R 2  = 0.42 and R 2  = 0.37., Conclusions: The set of case-mix adjustment variables for the EQ-5D at three months differed considerably from the set for clinical outcomes in stroke care. The case-mix adjustment variables that were specific to this PROM were sex, socio-economic status and nationality. These variables should be considered in future attempts to risk-adjust for PROMs during benchmarking of hospitals.

Background: Randomized controlled trials (RCTs) pose specific challenges in rare and heterogeneous neurological diseases due to the small numbers of patients and heterogeneity in disease course. Two analytical approaches have been proposed to optimally handle these issues in RCTs: covariate adjustment and ordinal analysis. We investigated the potential gain in efficiency of these approaches in rare and heterogeneous neurological diseases, using Guillain-Barré syndrome (GBS) as an example. Methods: We analyzed two published GBS trials with primary outcome ‘at least one grade improvement’ on the GBS disability scale. We estimated the treatment effect using logistic regression models with and without adjustment for prognostic factors. The difference between the unadjusted and adjusted estimates was disentangled in imbalance (random differences in baseline covariates between treatment arms) and stratification (change of the estimate due to covariate adjustment). Second, we applied proportional odds regression, which exploits the ordinal nature of the GBS disability score. The standard error of the estimated treatment effect indicated the statistical efficiency. Results: Both trials were slightly imbalanced with respect to baseline characteristics, which was corrected in the adjusted analysis. Covariate adjustment increased the estimated treatment effect in the two trials by 8% and 18% respectively. Proportional odds analysis resulted in lower standard errors indicating more statistical power. Conclusion: Covariate adjustment and proportional odds analysis most efficiently use the available data and ensure balance between the treatment arms to obtain reliable and valid treatment effect estimates. These approaches merit application in future trials in rare and heterogeneous neurological diseases like GBS. [ABSTRACT FROM AUTHOR]