Improved beta-binomial estimation for reliability of healthcare quality measures

BackgroundThe popular beta-binomial approach to estimate the reliability of healthcare quality measures (Adams et al. 2010New England Journal of Medicine) may yield grossly over-estimated reliabilities for providers with event rates equal to 0% or 100%.ObjectiveImprove the beta-binomial approach to yield more reasonable reliability estimates for providers with event rates equal to 0% or 100%.MethodWe revise the beta-binomial approach by substituting Bayesian estimates with various priors for the crude event rates. We evaluate the new reliability estimates using Monte Carlo studies and two real-world measure examples.Results and conclusionThe revised beta-binomial approach based on Jeffreys non-informative prior yields more reasonable reliability estimates for providers with event rates equal to 0% or 100% and statistically outperforms the original beta-binomial approach regarding bias and standard errors.