Inner workings of the Kenward–Roger test

For testing a linear hypothesis about fixed effects in a normal mixed linear model, a popular approach is to use a Wald test, in which the test statistic is assumed to have a null distribution that is approximately chi-squared. This approximation is questionable, however, for small samples. In 1997 Kenward and Roger constructed a test that addresses this problem. They altered the Wald test in three ways: (a) adjusting the test statistic, (b) approximating the null distribution by a scaled F distribution, and (c) modifying the formulas to achieve an exact F test in two special cases. Alterations (a) and (b) lead to formulas that are somewhat complicated but can be explained by using Taylor series approximations and a few convenient assumptions. The modified formulas used in alteration (c), however, are more mysterious. Restricting attention to models with linear variance–covariance structure, we provide details of a derivation that justifies these formulas. We show that similar but different derivations lead to different formulas that also produce exact F tests in the two special cases and are equally justifiable. A simulation study was done for testing the equality of treatment effects in block-design models. Tests based on the different derivations performed very similarly. Moreover, the simulations confirm that alteration (c) is worthwhile. The Kenward–Roger test showed greater accuracy in its p values than did the unmodified version of the test.