Search

The reporting of measures of effect size has become increasingly important in psychology. A Monte Carlo resampling permutation procedure is introduced to find near-optimum maximum values for Smart's τc measure for two-way ordinal contingency tables, also termed Kendall's τc since Kendall introduced τa and τb. Comparisons between resampling and exact procedures demonstrate the accuracy and utility of resampling measures of effect size for two-way ordinal contingency tables. The resampling procedure is shown to be more precise than the traditional method of standardizing τ,c. [ABSTRACT FROM AUTHOR]

The measurement of the magnitude of association between a nominal independent variable and an ordinal dependent variable is an important, but neglected, component in psychological research. Two measures of nominal-ordinal association are described and compared. Resampling permutation methods are utilized to compute probability values for both measures. [ABSTRACT FROM AUTHOR]

An exact permutation test is provided for the tetrachoric correlation coefficient. Comparisons with the conventional test employing Student's t distribution demonstrate the necessity of using the permutation approach for small sample sizes and/or disproportionate marginal frequency totals. [ABSTRACT FROM AUTHOR]

A permutation procedure to compute a resampling probability value and empirical quantile limits is provided for Cronbach's alpha measure of internal reliability. The underlying assumptions for the conventional use of alpha are discussed. The use of a permutation test for Cronbach's alpha is highlighted as a valuable tool when sample sizes are small and necessary assumptions cannot be met. [ABSTRACT FROM AUTHOR]

Five procedures to calculate the probability of weighted kappa with multiple raters under the null hypothesis of independence are described and compared in terms of accuracy, ease of use, generality, and limitations. The five procedures are (1) exact variance, (2) resampling contingency, (3) intraclass correlation, (4) randomized block, and (5) resampling block. While each procedure possesses strengths and limitations, the resampling contingency procedure is shown to be the most versatile and accurate of the five procedures, provided the number of raters is not too large. The resampling contingency procedure permits any weighting scheme, accommodates both symmetrical and asymmetrical weights, is suitable for both weighted and unweighted kappa, and makes no assumptions about either the data distribution or the probability distribution. [ABSTRACT FROM AUTHOR]

A permutation algorithm and associated FORTRAN program are provided for resampling weighted kappa. Program RWK provides the weighted kappa test statistic and the resampling one-sided upper-tail probability value. [ABSTRACT FROM AUTHOR]

A new procedure to compute weighted kappa with multiple raters is described. A resampling procedure to compute approximate probability values for weighted kappa with multiple raters is presented. Applications of weighted kappa are illustrated with an example analysis of classifications by three independent raters. [ABSTRACT FROM AUTHOR]

The number of resamplings necessary to accurately estimate a probability value is an open question. One million resamplings is shown to be sufficient to ensure precision to three places under most conditions. [ABSTRACT FROM AUTHOR]

Weighted kappa described by Cohen in 1968 is widely used in psychological research to measure agreement between two independent raters. Everitt then provided the exact variance for weighted kappa for two raters, in this paper, Everitt's exact variance is extended to three or more raters. [ABSTRACT FROM AUTHOR]

A resampling algorithm is presented for analyzing multiway contingency tables with fixed marginal frequency totals. Applications are illustrated with extensions of Fisher's exact, Pearson's chi-squared, and likelihood-ratio tests to three-way contingency tables. [ABSTRACT FROM AUTHOR]

Measures of effect size are increasingly important in psychological research. In this paper, a chance-corrected measure of effect size is introduced for Cochran's Q test. [ABSTRACT FROM AUTHOR]

An algorithm and associated FORTRAN program are provided for six common measures of ordinal association: Kendall's τa and τb, Stuart's τc, Goodman and Kruskal's γ, and Somers' dyx and dxy. Program ROMA reports the observed data table, the values for the six test statistics, and the resampling upper- and lower-tail probability values associated with each test statistic. [ABSTRACT FROM AUTHOR]

In the vast majority of psychological research utilizing multiple regression analysis, asymptotic probability values are reported. This paper demonstrates that asymptotic estimates of standard errors provided by multiple regression are not always accurate. A resampling permutation procedure is used to estimate the standard errors. In some cases the results differ substantially from the traditional least squares regression estimates. [ABSTRACT FROM AUTHOR]

A multivariate permutation test of similarity between two populations with corresponding unordered disjoint categories is described. The test statistic, resampling probability value, and measure of effect size are described. [ABSTRACT FROM AUTHOR]

Goodman and Kruskal's τ measure of categorical association is advanced as a replacement for conventional measures of effect size for r × c contingency tables. Goodman and Kruskal's τ is an asymmetric measure of categorical association which is based entirely on the observed data and possesses a clear interpretation in terms of proportional reduction in error. Comparisons with conventional measures of effect size based on chi-squared such as Pearson's ϕ², Tschuprow's T², and Cramér's V² demonstrate the advantages of employing τ as a measure of effect size. [ABSTRACT FROM AUTHOR]

Permutation procedures to compute exact and resampling probability values associated with measures of association for ordered r × c contingency tables are described. Comparisons with asymptotic probability values demonstrated that exact and resampling permutation procedures were preferred for sparse contingency tables. [ABSTRACT FROM AUTHOR]

A permutation method is presented to calculate resampling probability values for differences between two independent indices of ordinal variation and consensus. [ABSTRACT FROM AUTHOR]

Performance of quantile rank score tests used for hypothesis testing and constructing confidence intervals for linear quantile regression estimates (0?=?t?=?1) were evaluated by simulation for models with p ?=?2 and 6 predictors, moderate collinearity among predictors, homogeneous and hetero-geneous errors, small to moderate samples ( n ?=?20–300), and central to upper quantiles (0.50–0.99). Test statistics evaluated were the conventional quantile rank score T statistic distributed as ? 2 random variable with q degrees of freedom (where q parameters are constrained by H 0 :) and an F statistic with its sampling distribution approximated by permutation. The permutation F -test maintained better Type I errors than the T -test for homogeneous error models with smaller n and more extreme quantiles t. An F distributional approximation of the F statistic provided some improvements in Type I errors over the T -test for models with?>?2 parameters, smaller n , and more extreme quantiles but not as much improvement as the permutation approximation. Both rank score tests required weighting to maintain correct Type I errors when heterogeneity under the alternative model increased to 5 standard deviations across the domain of X . A double permutation procedure was developed to provide valid Type I errors for the permutation F -test when null models were forced through the origin. Power was similar for conditions where both T - and F -tests maintained correct Type I errors but the F -test provided some power at smaller n and extreme quantiles when the T -test had no power because of excessively conservative Type I errors. When the double permutation scheme was required for the permutation F -test to maintain valid Type I errors, power was less than for the T -test with decreasing sample size and increasing quantiles. Confidence intervals on parameters and tolerance intervals for future predictions were constructed based on test inversion for an example application relating trout densities to stream channel width:depth. [ABSTRACT FROM AUTHOR]

An algorithm and associated FORTRAN program are provided for the exact variance of weighted kappa. Program VARKAP provides the weighted kappa test statistic, the exact variance of weighted kappa, a Z score, one-sided lower- and upper-tail N(0,1) probability values, and the two-tail N(0,1) probability value. [ABSTRACT FROM AUTHOR]

When analyzing categorical data, it is often important to assess the magnitude of variation, or consensus, among observations in unordered categories. Utilizing the theory of partitions, exact solutions for five commonly used measures of categorical variation are presented. When the number of partitions is very large, resampling methods provide close approximations to exact probability values. [ABSTRACT FROM AUTHOR]

Fisher's well-known continuous method for combining independent probability values from continuous distributions is compared with an exact discrete analog of Fisher's continuous method for combining independent probability values from discrete distributions using matched-pairs t-test data. Fisher's continuous method is shown to be inadequate for combining probability values from many discrete distributions, given the continuity assumption when discrete distributions are considered. Although Fisher's continuous method does not detect a well-documented effect among distributions, the exact discrete analog method clearly detects the effect. [ABSTRACT FROM AUTHOR]

Resampling permutation procedures provide good approximations to exact permutation procedures and are therefore the preferred alternative when large samples render exact tests intractable. Resampling permutation procedures are described for three measures of variation and the three corresponding measures of consensus among ordered categories. [ABSTRACT FROM AUTHOR]

Permutation procedures to compute exact and resampling probability values for weighted kappa are described. Comparisons with asymptotic probability values demonstrate that exact permutation procedures are advantageous for sparse data sets, whereas resampling permutation procedures are appropriate for both sparse and nonsparse data sets. [ABSTRACT FROM AUTHOR]

Permutation tests are based on all possible arrangements of observed data sets. Consequently, such tests yield exact probability values obtained from discrete probability distributions. An exact nondirectional method to combine independent probability values that obey discrete probability distributions is introduced. The exact method is the discrete analog to Fisher's classical method for combining probability values from independent continuous probability distributions. If the combination of probability values includes even one probability value that obeys a sparse discrete probability distribution, then Fisher's classical method may be grossly inadequate. [ABSTRACT FROM AUTHOR]

An algorithm and computer program to calculate exact goodness-of-fit tests for unordered categories with equal probabilities under the null hypothesis are presented. FORTRAN program EBGF utilizes partitions and multiinomial weights to reduce computation times for Fisher's exact, exact chi-square, exact likelihood-ratio, exact Freeman-Tukey, and exact Cressie-Read goodness-of-fit tests. [ABSTRACT FROM AUTHOR]

A recent trend in the psychological literature has been to include measures of effect size when reporting probability values. The several measures of effect size associated with the Student t test for two independent samples are appropriate only when the variances are homogeneous. In this paper, commonly used measures of effect size arc considered and compared, using four data sets. A chance-corrected measure of effect size is provided for two or more treatment groups characterized by either homogeneous or heterogeneous variances. [ABSTRACT FROM AUTHOR]

Discusses the longitudinal analysis of data with multiple binary category choices. Description of one exact and two approximate permutation methods; Analysis of binary answers to multiple-response questions; Provision of probabilities under the null hypothesis.

An extension of a multiple regression prediction model to multiple response variables is presented. An algorithm using least sum of Euclidean distances between the multivariate observed and model-predicted response values provides regression coefficients, a measure of effect size, and inferential procedures for evaluating the extended multivariate multiple regression prediction model. [ABSTRACT FROM AUTHOR]

The Fisher-Pitman and powers of ranks permutation tests are shown to provide substantial resistance to extreme values when compared with the conventional t test for analyzing matched-pairs experimental designs. [ABSTRACT FROM AUTHOR]

A universal measure of the magnitude of effect size is introduced for Euclidean-based regression analyses of experimental designs. The measure is based on the ratio of the sums of Euclidean residuals obtained under the alternative and null hypotheses. [ABSTRACT FROM AUTHOR]

A data-dependent analysis assumes that all the information available to a researcher is contained within the observed data. Data-dependent methods for the analysis of experimental designs are shown to provide significant advantages over conventional techniques such as an F test. Two versions of three data-dependent methods based on permutations of the data are described and compared. One version utilizes ordinary least squares regression, and the other version utilizes least absolute deviations regression to analyze experimental designs. Analyses of an unbalanced twoway experimental design illustrate the differences among the six data-dependent approaches and the classical ordinary least squares F test, which depends on the assumptions of normality, homogeneity, and independence. [ABSTRACT FROM AUTHOR]

A nonasymptotic chi-squared technique is shown to have very useful properties for the analysis of large sparse r-way contingency tables. Examples of analyses of 4 × 5, 5 × 6, 6 × 7, and two 2 × 2 × 2 sparse contingency tables provide comparisons of the nonasymptotic chi-squared technique with asymptotic chi-squared and exact chi-squared techniques. The asymptotic chi-squared analyses yield inflated probability values for the five tables. The nonasymptotic chi-squared technique yields probability values much closer to the exact probability values than the asymptotic chisquared technique for the five tables. [ABSTRACT FROM AUTHOR]

Presents a multivariate multiple regression analysis. Use of Euclidean-distance permutation procedure in the evaluation of multivariate residuals; Application of permutation method for linear models; Assumptions on the randomization of subjects to treatment.

Investigates the differences between kappa agreement coefficients of Cohen and Brennan and Prediger. Types of interaction; Measurements of alienation; Agreement between observers and their interpretations.

Examines the precision and accuracy of the Fisher Z transform for a variety of bivariate distributions, sample sizes and population parameters. General applications of the Fisher Z transform; Description of the bivariate distributions, confidence intervals and hypothesis testing; Propriety of uncritically using the Fisher Z transform in actual research.

Describes the exact and Monte Carlo resampling FORTRAN programs for the Wilcoxon-Mann-Whitney rank-sum test and the Kruskal-Wallis one-way analysis of variance for ranks test. Compensation for tied values by program algorithms and non-dependence on asymptotic approximations for probability values.

Describes the FORTRAN program RTERP, an alternative ordered hypothesis to obtain the probability estimate from a randomization routine where a large sample of all possible arrangements of the data are considered. Concept of the program; Applications.

Presents an asymmetrical alternative to the chi square test of homogeneity. Advantages of an asymmetrical test of homogeneity of proportions over a symmetrical test; Use of product multinomial sampling; Description of the Pearson X2 test of homogeneity.

Analyzes the residuals obtained from least sum of absolute deviations regression. Applications to a split-plot design; Applicability of the procedure; Example applications of the permutation analysis to block designs and randomized designs.

Analyzes residuals obtained from least sum of absolute deviations regression. Use of procedure consistent with Euclidean geometry; Applications to one-way randomized designs, one-way randomized designs with covariate and factorial designs.

Analyzes residuals obtained from least sum of absolute deviations regressions by a procedure consistent with Euclidean geometry. Illustration and evaluation of applications to a one-way block design, a two-way block design and a Latin square design.

Discusses the use of unbiased capture-recapture estimators for proving both direct and inverse sampling without replacement. Importance of estimation of a target population to estimates for funding and sampling; Assumptions necessary for the determination of estimators.

Discusses the use of dummy coding to test main effects in factorial designs of psychological research. Creation of a design matrix of coded vectors; Illustrative example.

Presents information pertaining to the use of a FORTRAN program to analyze the results of experimental designs. Use of a Euclidean distance permutation procedure to evaluate residuals; Applications for the program.

Introduces a permutation alternative for Hotelling's multivariate matched-pair T2 test. Allowance for analyses when the number of subjects is less than or equal to the number of measurements; Improved discrimination over Hotelling's multivariate matched-pair T2 test.

Proposes an alternative method of analysis for the semantic differential which conforms to the procedure initially advanced by Osgood and colleagues in 1957. Problem of an appropriate distance function; Permutation multivariate repeated-measures analysis of variance; Euclidean data space.

Presents a Fortran program to calculate exact probabilities for first- and second-order interactions in 2x2x2 contingency tables with fixed marginals. Use of an arbitrary constant for the initial table and recursively defined values for all subsequent tables; Logic on which the program is based; Program language.

Describes a program called subroutine MULT that computes the exact cumulative probabilities for the multinomial distribution, reported by researchers from Brown University, University of Minnesota, State University of Tony Brook and the University of Nevada. Application; Availability.