PROBLEMS WITH BINOMIAL TWO-SIDED TESTS AND THE ASSOCIATED CONFIDENCE INTERVALS

Summary Confidence intervals for parameters of distributions with discrete sample spaces will be less conservative (i.e. have smaller coverage probabilities that are closer to the nominal level) when defined by inverting a test that does not require equal probability in each tail. However, the P-value obtained from such tests can exhibit undesirable properties, which in turn result in undesirable properties in the associated confidence intervals. We illustrate these difficulties using P-values for binomial proportions and the difference between binomial proportions.