Search

The increase in the squared multiple correlation coefficient ([delta]R[superscript 2]) associated with a variable in a regression equation is a commonly used measure of importance in regression analysis. Algina, Keselman, and Penfield found that intervals based on asymptotic principles were typically very inaccurate, even though the sample size was quite large (i.e., larger than 200). However, they also reported that probability coverage for the confidence intervals based on a bootstrap method was typically quite accurate, and moreover, this accuracy was obtained with relatively small sample sizes with six or fewer predictors. They further speculated that nonnormality would likely affect the accuracy of interval coverage. In the present study, the authors investigated the accuracy of coverage probability for confidence intervals obtained by using asymptotic and percentile bootstrap methodology when either predictors, residuals, or both are nonnormal. Coverage probability for asymptotic confidence intervals is poor, but adequate coverage probability can be obtained with reasonable sample sizes by using percentile bootstrap methodology. As well, the authors found that the width of these intervals was relatively precise (i.e., narrow) for the larger cases of sample size investigated. (Contains 7 tables.)

Applications of distribution theory for the squared multiple correlation coefficient and the squared cross-validation coefficient are reviewed, and computer programs for these applications are made available. The applications include confidence intervals, hypothesis testing, and sample size selection. (Contains 2 tables.)

The increase in the squared multiple correlation coefficient ([Delta]R[squared]) associated with a variable in a regression equation is a commonly used measure of importance in regression analysis. The coverage probability that an asymptotic and percentile bootstrap confidence interval includes [Delta][rho][squared] was investigated. As expected, coverage probability for the asymptotic confidence interval was often inadequate (outside the interval 0.925 to 0.975 for a 95% confidence interval), even when sample size was quite large (i.e., 200). However, adequate coverage probability for the confidence interval based on a bootstrap interval could typically be obtained with a sample size of 200 or less, and moreover, this accuracy was obtained with relatively small sample sizes (100 or less) with six or fewer predictors. (Contains 4 tables.)

Kelley compared three methods for setting a confidence interval (CI) around Cohen's standardized mean difference statistic: the noncentral-"t"-based, percentile (PERC) bootstrap, and biased-corrected and accelerated (BCA) bootstrap methods under three conditions of nonnormality, eight cases of sample size, and six cases of population effect size (ES) magnitude. Kelley recommended the BCA bootstrap method. The authors expand on his investigation by including additional cases of nonnormality. Like Kelley, they find that under many conditions, the BCA bootstrap method works best; however, they also find that in some cases of nonnormality, the method does not control probability coverage. The authors also define a robust parameter for ES and a robust sample statistic, based on trimmed means and Winsorized variances, and cite evidence that coverage probability for this parameter is good over the range of nonnormal distributions investigated when the PERC bootstrap method is used to set CIs for the robust ES. (Contains 7 tables.)

One approach to the analysis of repeated measures data allows researchers to model the covariance structure of their data rather than presume a certain structure, as is the case with conventional univariate and multivariate test statistics. This mixed-model approach, available through SAS PROC MIXED, was compared to a Welch-James type statistic. The Welch-James approach is known to provide generally robust tests of treatment effects in a repeated measures between-by within-subjects design under assumption violations given certain sample size requirements. The mixed-model "F" tests were based on Kenward-Roger's adjusted degrees of freedom solution, an approach specifically proposed for small sample settings. The authors investigated Type I error control for repeated measures main and interaction effects in unbalanced designs when normality and covariance homogeneity assumptions did not hold. The mixed-model Kenward-Roger's adjusted "F" tests showed superior Type I error control in small sample size conditions in which the Welch-James type statistic was nonrobust; power rates, however, did not favor one approach over the other.

The approximate degrees of freedom Welch-James (WJ) and Brown-Forsythe (BF) procedures for testing within-subjects effects in multivariate groups by trials repeated measures designs were investigated under departures from covariance homogeneity and normality. Empirical Type I error and power rates were obtained for least-squares estimators and robust estimators based on trimming/Winsorization. The BF and WJ procedures with least-squares estimators produced comparable Type I error rates for the main effect. For the interaction, the former could become conservative, while the latter was liberal when sample sizes were small. When robust estimators were adopted, the WJ test generally controlled the error rate while the BF test was conservative. Both procedures had similar power for the main effect test, while the WJ procedure was more powerful for tests of the interaction when covariances were heterogeneous.

Discusses the squared cross-validity coefficient as a measure of the predictive validity of a sample linear prediction equation. Presents sample-size tables that should result in very small discrepancies between the squared multiple correlation and the squared cross-validity correction, thus facilitating the selection of sample size for predictive studies. (Author/SLD)

Examines the recommendations of H. Keselman, K. Carriere, and L. Lix (1993) regarding choice of sample size for obtaining robust tests of the repeated measures main and interaction hypotheses in a one Between-Subjects by one Within- Subjects design with a Welch-James type multivariate test when covariance matrices are heterogeneous. (SLD)

The power to detect main and interaction effects in a factorial design was determined when the Bonferroni method was used to control the overall rate of Type I error. For sample sizes typical of educational research, the power of this procedure was considerably less than that of recommended standards. (TJH)