Search

When a statistically significant mean difference is found, the magnitude of the difference is judged qualitatively using an effect size such as Cohen's d. In contrast, in a structural equation model (SEM), the result of the statistical test of model fit is often disregarded if significant, and inferences are drawn using "close" models retained based on point estimates of sample statistics (goodness-of-fit indices). However, when a SEM cannot be retained using a test of exact fit, all substantive inferences drawn from it are suspect. It is therefore important to determine the size of the model misfit. Standardized residual covariances and residual correlations provide standardized effect sizes of the misfit of SEM models. They can be summarized using the Standardized Root Mean Squared Residual (SRMSR) and the Correlation Root Mean Squared Residual (CRMSR) which can be used as overall effect sizes of the misfit. Statistical theory is provided that allows the construction of confidence intervals and tests of close fit based on the SRMSR and CRMSR. It is hoped that the use of standardized effect sizes of misfit will help reconcile current practices in SEM and elsewhere in statistics.

The relationship between linear factor models and latent profile models is addressed within the context of maximum likelihood estimation based on the joint distribution of the manifest variables. Although the two models are well known to imply equivalent covariance decompositions, in general they do not yield equivalent estimates of the unconditional covariances. In particular, a 2-class latent profile model with Gaussian components underestimates the observed covariances but not the variances, when the data are consistent with a unidimensional Gaussian factor model. In explanation of this phenomenon we provide some results relating the unconditional covariances to the goodness of fit of the latent profile model, and to its excess multivariate kurtosis. The analysis also leads to some useful parameter restrictions related to symmetry.

Maydeu-Olivares and Joe (J. Am. Stat. Assoc. 100:1009–1020, 2005; Psychometrika 71:713–732, 2006) introduced classes of chi-square tests for (sparse) multidimensional multinomial data based on low-order marginal proportions. Our extension provides general conditions under which quadratic forms in linear functions of cell residuals are asymptotically chi-square. The new statistics need not be based on margins, and can be used for one-dimensional multinomials. We also provide theory that explains why limited information statistics have good power, regardless of sparseness. We show how quadratic-form statistics can be constructed that are more powerful than X2 and yet, have approximate chi-square null distribution in finite samples with large models. Examples with models for truncated count data and binary item response data are used to illustrate the theory.

Recently, there has been increasing interest in reporting subscores. This paper examines reporting of subscores using multidimensional item response theory (MIRT) models (e.g., Reckase in Appl. Psychol. Meas. 21:25–36, 1997; C.R. Rao and S. Sinharay (Eds), Handbook of Statistics, vol. 26, pp. 607–642, North-Holland, Amsterdam, 2007; Beguin & Glas in Psychometrika, 66:471–488, 2001). A MIRT model is fitted using a stabilized Newton–Raphson algorithm (Haberman in The Analysis of Frequency Data, University of Chicago Press, Chicago, 1974; Sociol. Methodol. 18:193–211, 1988) with adaptive Gauss–Hermite quadrature (Haberman, von Davier, & Lee in ETS Research Rep. No. RR-08-45, ETS, Princeton, 2008). A new statistical approach is proposed to assess when subscores using the MIRT model have any added value over (i) the total score or (ii) subscores based on classical test theory (Haberman in J. Educ. Behav. Stat. 33:204–229, 2008; Haberman, Sinharay, & Puhan in Br. J. Math. Stat. Psychol. 62:79–95, 2008). The MIRT-based methods are applied to several operational data sets. The results show that the subscores based on MIRT are slightly more accurate than subscore estimates derived by classical test theory.

When using linear models for cluster-correlated or longitudinal data, a common modeling practice is to begin by fitting a relatively simple model and then to increase the model complexity in steps. New predictors might be added to the model, or a more complex covariance structure might be specified for the observations. When fitting models for binary or ordered-categorical outcomes, however, comparisons between such models are impeded by the implicit rescaling of the model estimates that takes place with the inclusion of new predictors and/or random effects. This paper presents an approach for putting the estimates on a common scale to facilitate relative comparisons between models fit to binary or ordinal outcomes. The approach is developed for both population-average and unit-specific models.

Diagnostic classification models are confirmatory in the sense that the relationship between the latent attributes and responses to items is specified or parameterized. Such models are readily interpretable with each component of the model usually having a practical meaning. However, parameterized diagnostic classification models are sometimes too simple to capture all the data patterns, resulting in significant model lack of fit. In this paper, we attempt to obtain a compromise between interpretability and goodness of fit by regularizing a latent class model. Our approach starts with minimal assumptions on the data structure, followed by suitable regularization to reduce complexity, so that readily interpretable, yet flexible model is obtained. An expectation–maximization-type algorithm is developed for efficient computation. It is shown that the proposed approach enjoys good theoretical properties. Results from simulation studies and a real application are presented.

Discretized multivariate normal structural models are often estimated using multistage estimation procedures. The asymptotic properties of parameter estimates, standard errors, and tests of structural restrictions on thresholds and polychoric correlations are well known. It was not clear how to assess the overall discrepancy between the contingency table and the model for these estimators. It is shown that the overall discrepancy can be decomposed into a distributional discrepancy and a structural discrepancy. A test of the overall model specification is proposed, as well as a test of the distributional specification (i.e., discretized multivariate normality). Also, the small sample performance of overall, distributional, and structural tests, as well as of parameter estimates and standard errors is investigated under conditions of correct model specification and also under mild structural and/or distributional misspecification. It is found that relatively small samples are needed for parameter estimates, standard errors, and structural tests. Larger samples are needed for the distributional and overall tests. Furthermore, parameter estimates, standard errors, and structural tests are surprisingly robust to distributional misspecification.

Complete response vectors of all answer options in multiple-choice items can be used to estimate ability. The rising selection ratios criterion is necessary for scoring individuals because it implies that estimated ability always increases when the correct alternative is selected. This paper introduces the generalized DLT model, which assumes rising selection ratios and uses three parameters to describe each incorrect alternative. It is shown that the nominal categories and the Thissen and Steinberg model do not meet the criterion. A simulation study on goodness of recovery and an example with real data are also included.

The purpose of this study was to investigate and compare the performance of a stepwise variable selection algorithm to traditional exploratory factor analysis. The Monte Carlo study included six factors in the design; the number of common factors; the number of variables explained by the common factors; the magnitude of factor loadings; the number of variables not explained by the common factors; the type of anomaly evidenced by the poorly explained variables; and sample size. The performance of the methods was evaluated in terms of selection and pattern accuracy, and bias and root mean squared error of the structure coefficients. Results indicate that the stepwise algorithm was generally ineffective at excluding anomalous variables from the factor model. The poor selection accuracy of the stepwise approach suggests that it should be avoided.

A taxonomy of latent structure assumptions (LSAs) for probability matrix decomposition (PMD) models is proposed which includes the original PMD model (Maris, De Boeck, & Van Mechelen, 1996) as well as a three-way extension of the multiple classification latent class model (Maris, 1999). It is shown that PMD models involving different LSAs are actually restricted latent class models with latent variables that depend on some external variables. For parameter estimation a combined approach is proposed that uses both a mode-finding algorithm (EM) and a sampling-based approach (Gibbs sampling). A simulation study is conducted to investigate the extent to which information criteria, specific model checks, and checks for global goodness of fit may help to specify the basic assumptions of the different PMD models. Finally, an application is described with models involving different latent structure assumptions for data on hostile behavior in frustrating situations.

A Monte Carlo algorithm realizing a family of nonparametric tests for the Rasch model is introduced which are conditional on the item and subject marginals. The algorithm is based on random changes of elements of data matrices without changing the marginals; most powerful tests against all alternative hypotheses are given for which a monotone characteristic may be computed from the data matrix; alternatives may also be composed. Computation times are long, but exactp-values are approximated with the quality of approximation only depending on calculation time, but not on the number of persons. The power and the flexibility of the procedure is demonstrated by means of an empirical example where, among others, indicators for increased item similarities, the existence of subscales, violations of sufficiency of the raw score as well as learning processes were found. Many of the features described are implemented in the program T-Rasch 1.0 by Ponocny and Ponocny-Seliger (1999).

It is very important to choose appropriate variables to be analyzed in multivariate analysis when there are many observed variables such as those in a questionnaire. What is actually done in scale construction with factor analysis is nothing but variable selection.

In this paper it will be shown that a certain class of constrained latent class models may be interpreted as a special case of nonparametric multidimensional item response models. The parameters of this latent class model will be estimated using an application of the Gibbs sampler. It will be illustrated that the Gibbs sampler is an excellent tool if inequality constraints have to be taken into consideration when making inferences. Model fit will be investigated using posterior predictive checks. Checks for manifest monotonicity, the agreement between the observed and expected conditional association structure, marginal local homogeneity, and the number of latent classes will be presented.

Loglinear unidimensional and multidimensional Rasch models are considered for the analysis of repeated observations of polytomous indicators with ordered response categories. Reparameterizations and parameter restrictions are provided which facilitate specification of a variety of hypotheses about latent processes of change. Models of purely quantitative change in latent traits are proposed as well as models including structural change. A conditional likelihood ratio test is presented for the comparison of unidimensional and multiple scales Rasch models. In the context of longitudinal research, this renders possible the statistical test of homogeneity of change against subject-specific change in latent traits. Applications to two empirical data sets illustrate the use of the models.

Using the item response model as developed on the multinomial distribution, asymptotic variances are obtained for residuals associated with response patterns and first-, and second-order marginal frequencies of manifest variables. When the model does not fit well, an examination of these residuals may reveal the source of the poor fit. Finally, a limited-information test of fit for the model is developed by using residuals defined for the first-, and second-order marginals. Model evaluation based on residuals for these marginals is particularly useful when the response pattern frequencies are sparse.

Residuals for check of model fit in the polytomous Rasch model are examined. Comparisons are made between using counts for all response pattern and using item totals for score groups for the construction of the residuals. Comparisons are also, for the residuals based on score group totals, made between using as basis the item totals, or using the estimated item parameters. The developed methods are illustrated by two examples, one from a psychiatric rating scale, one from a Danish Welfare Study.

A loglinear IRT model is proposed that relates polytomously scored item responses to a multidimensional latent space. The analyst may specify a response function for each response, indicating which latent abilities are necessary to arrive at that response. Each item may have a different number of response categories, so that free response items are more easily analyzed. Conditional maximum likelihood estimates are derived and the models may be tested generally or against alternative loglinear IRT models.

Takane and Sergent developed a model (MAXRT) for scaling same/different judgments and response times (RTs) simultaneously. The model assumes that RTs are distributed lognormally. Our experiment showed that the RT distribution of the judgments might be task dependent. It is shown that lognormal RTs provide a far better fit than exponential, normal, and Pareto distributed RTs (with the same means and variances), but that the final parameter estimates from the data set with lognormal RTs hardly differ from the alternatively distributed RTs. Finally, despite the robustness of the distributional assumption of the RTs with respect to the parameter estimates, it is shown that RTs have an informational value that is not contained in the same/different judgments alone.

We investigate the performance of three statistics, R 1, R 2 (Glas in Psychometrika 53:525–546, 1988), and M 2 (Maydeu-Olivares & Joe in J. Am. Stat. Assoc. 100:1009–1020, 2005, Psychometrika 71:713–732, 2006) to assess the overall fit of a one-parameter logistic model (1PL) estimated by (marginal) maximum likelihood (ML). R 1 and R 2 were specifically designed to target specific assumptions of Rasch models, whereas M 2 is a general purpose test statistic. We report asymptotic power rates under some interesting violations of model assumptions (different item discrimination, presence of guessing, and multidimensionality) as well as empirical rejection rates for correctly specified models and some misspecified models. All three statistics were found to be more powerful than Pearson’s X 2 against two- and three-parameter logistic alternatives (2PL and 3PL), and against multidimensional 1PL models. The results suggest that there is no clear advantage in using goodness-of-fit statistics specifically designed for Rasch-type models to test these models when marginal ML estimation is used.

This paper presents a nonspatial operationalization of the Krumhansl (1978, 1982) distancedensity model of similarity. This model assumes that the similarity between two objectsi andj is a function of both the interpoint distance betweeni andj and the density of other stimulus points in the regions surroundingi andj. We review this conceptual model and associated empirical evidence for such a specification. A nonspatial, tree-fitting methodology is described which is sufficiently flexible to fit a number of competing hypotheses of similarity formation. A sequential, unconstrained minimization algorithm is technically presented together with various program options. Three applications are provided which demonstrate the flexibility of the methodology. Finally, extensions to spatial models, three-way analyses, and hybrid models are discussed.

The partial credit model, developed by Masters (1982), is a unidimensional latent trait model for responses scored in two or more ordered categories. In the present paper some extensions of the model are presented. First, a marginal maximum likelihood estimation procedure is developed which allows for incomplete data and linear restrictions on both the item and the population parameters. Secondly, two statistical tests for evaluating model fit are presented: the former test has power against violation of the assumption about the ability distribution, the latter test offers the possibility of identifying specific items that do not fit the model.

The present paper is concerned with testing the fit of the Rasch model. It is shown that this can be achieved by constructing functions of the data, on which model tests can be based that have power against specific model violations. It is shown that the asymptotic distribution of these tests can be derived by using the theoretical framework of testing model fit in general multinomial and product-multinomial models. The model tests are presented in two versions: one that can be used in the context of marginal maximum likelihood estimation and one that can be applied in the context of conditional maximum likelihood estimation.

Thesmall-world problem revolves around the tracing of a line of acquaintances linking any two persons chosen at random. This paper discusses the statistical analysis of data arising out of two experimental studies of the small-world problem. The estimation of parameters in two previously proposed models is discussed, and the goodness-of-fit of these models is considered.

Although several goodness of fit tests have been developed for the Rasch model for dichotomous items, most of them are of a global, asymptotic, and confirmatory type. This paper, based on ideas from a recent thesis by Van den Wollenberg, offers some suggestions for local, small sample, and exploratory techniques: difficulty plots for person groups scoring right and wrong on a specific item, a slope test per item based on a binomial distribution per score group, and a unidimensionality check based on an extended hypergeometric distribution per score group.

McNemar's problem concerns the hypothesis of equal probabilities for the unlike pairs of correlated binary variables.

The properties of nonmetric multidimensional scaling (NMDS) are explored by specifying statistical models, proving statistical consistency, and developing hypothesis testing procedures. Statistical models with errors in the dependent and independent variables are described for quantitative and qualitative data. For these models, statistical consistency often depends crucially upon how error enters the model and how data are collected and summarized (e.g., by means, medians, or rank statistics). A maximum likelihood estimator for NMDS is developed, and its relationship to the standard Shepard-Kruskal estimation method is described. This maximum likelihood framework is used to develop a method for testing the overall fit of the model.

A general question is raised concerning the possible consequences of employing the very popular INDSCAL multidimensional scaling model in cases where the assumptions of that model may be violated. Simulated data are generated which violate the INDSCAL assumption that all individuals perceive the dimensions of the common object space to be orthogonal. INDSCAL solutions for these various sets of data are found to exhibit extremely high goodness of fit, but systematically distorted object spaces and negative subject weights. The author advises use of Tucker's three-mode model for multidimensional scaling, which can account for non-orthogonal perceptions of the object space dimensions. It is shown that the INDSCAL model is a special case of the three-mode model.

PINDIS, as recently presented by Lingoes and Borg [1978] not only marks the latest development within the scope of individual differences scaling, but, may be of benefit in some closely related topics, such as target analysis. Decisions on whether the various models available from PINDIS fit fallible data are relatively arbitrary, however, since a statistical theory of the fit measures is lacking. Using Monte Carlo simulation, expected fit measures as well as some related statistics were therefore obtained by scaling sets of 4(4)24 random configurations of 5(5)30 objects in 2, 3, and 4 dimensions (individual differences case) and by fitting one random configuration to a fixed random target for 5(5)30 objects in 2, 3, and 4 dimensions (target analysis case). Applications are presented.

A latent class model for rating data is presented which is the analogue of Andrich's binomial Rasch model for Lazarsfeld's latent class analysis (LCA). The response probabilities for the rating categories follow a binomial distribution and depend on class-specific item parameters. The EM-algorithm for parameter estimation as well as goodness of fit tests for the model are described. An example using questionnaire items on interest in physics illustrates the use of the model as an alternative to the latent trait approach of analyzing test data.

A new method is proposed for a simultaneous factor analysis of dichotomous responses from several groups of individuals. The method makes it possible to compare factor loading pattern, factor variances and covariances, and factor means over groups. The method uses information from first and second order proportions and estimates the model by generalized least-squares. Hypotheses regarding different degrees of invariance over groups may be evaluated by a large-sample chi-square test.

A Monte Carlo study assessed the effect of sampling error and model characteristics on the occurrence of nonconvergent solutions, improper solutions and the distribution of goodness-of-fit indices in maximum likelihood confirmatory factor analysis. Nonconvergent and improper solutions occurred more frequently for smaller sample sizes and for models with fewer indicators of each factor. Effects of practical significance due to sample size, the number of indicators per factor and the number of factors were found for GFI, AGFI, and RMR, whereas no practical effects were found for the probability values associated with the chi-square likelihood ratio test.

A model of Bradley-Terry type for paired comparisons is considered. In addition to the usual parameters, the model allows for individual parameters and corresponding parameters for the choice-alternatives. The model is applied to a set of data from a Danish investigation of the attitude of blue collar workers towards alternative social gains. The proposed model is finally compared to a model recently suggested by Schonemann and Wang.

A probabilistic model for the validation of behavioral hierarchies is presented. Estimation is by means of iterative convergence to maximum likelihood estimates, and two approaches to assessing the fit of the model to sample data are discussed. The relation of this general probabilistic model to other more restricted models which have been presented previously is explored and three cases of the general model are applied to exemplary data.

In problem solving situations, it has been suggested the superiority of groups over individuals is due simply to the fact that groups consist of several individuals. In this paper, the Lorge and Solomon [1955] approach to such situations is reexamined using the method of maximum likelihood. Extensions to trichotomous response situations are also presented, and the resulting models are applied to data gathered by Staub [1970]. The partitioning of the likelihood ratio goodness-of-fit statistic is then discussed in order that the effects of additional variables on the response variate can be assessed. Finally, the small sample behavior of the likelihood ratio statistic is examined.

A three-component model for comparative judgment which allows for individual differences in preference is proposed. An implication of the model is that errors in the observed proportions due to sampling individuals in paired comparisons experiments are correlated. By neglecting this correlation, Mosteller's test for the method of paired comparisons tends to accept falsely the goodness of fit of the Case V solution. It is shown that bounds may be set for the correlation effect which make a valid test possible in some cases and provide useful standard errors for the estimated affective values.

The Rasch model is an item analysis model with logistic item characteristic curves of equal slope,i.e. with constant item discriminating powers. The proposed goodness of fit test is based on a comparison between difficulties estimated from different scoregroups and over-all estimates. Based on the within scoregroup estimates and the over-all estimates of item difficulties a conditional likelihood ratio is formed. It is shown that—2 times the logarithm of this ratio isx 2-distributed when the Rasch model is true. The power of the proposed goodness of fit test is discussed for alternative models with logistic item characteristic curves, but unequal discriminating items from a scholastic aptitude test.

A stochastic process applicable to the learning behavior of an individual subject is discussed. The process describes both the response times and the sequence of choices obtained from a situation involving two alternatives. Parameter estimates and techniques for assessing goodness of fit are considered.

Maximum likelihood estimates of item parameters of a scholastic aptitude test were computed using the normal and logistic models. The goodness of fit of ogives specified by the pairs of item parameters to the observed data was determined for all items. While negligible differences in the limen values were found, differences in item discrimination indices indicated that interpretation of these indices requires separate frames of reference. The empirical results showed the logistic model to be a useful alternative to the normal model in item analysis.

A theory of cue learning, which gives rise to a system of recurrent events in Feller's sense, is analyzed mathematically. The distribution of total errors and sampling distribution of mean errors are derived, and the learning curve is investigated. Maximum likelihood estimates of parameters and sampling variances of those estimates are derived. Likelihood ratio tests of the usual null hypotheses and approximate tests of goodness of fit of substantive hypotheses are developed. The distinguishing characteristic of these tests is that they are concerned with meaningful parameters of the learning process.

The problem of dependence of the nonnormed fit index on sample size in covariance structure analysis is discussed. Contrary to Bollen (1986) we show that the mean of the nonnormed fit index is independent of sample size for true and almost true models whereas Bollen's alternative index does depend on sample size.

The sampling properties of four item discrimination indices (biserialr, Cook's indexB, theU–L 27 per cent index, and DeltaP) were investigated in order to ascertain their sampling properties when small samples drawn from actual test data rather than constructed data were employed. The empirical results indicated that the mean index values approximated the population values and that values of the standard deviations computed from large sample formulas were good approximations to the standard deviations of the observed distributions based on samples of size 120 or less. Goodness of fit tests comparing the observed distributions with the corresponding distribution of the product-moment correlation coefficient based upon a bivariate normal population indicated that this correlational model was inappropriate for the data. The lack of adequate mathematical models for the sampling distributions of item discrimination indices suggests that one should avoid indices whose only real reason for existence was the simplification of computational procedures.

Classical item analysis procedures were developed for dichotomously scored items and do not apply to items allowing multiple correct responses. Maximum likelihood procedures analogous to those employed in polychotomous bio-assay are presented which yield estimates of the sets of parameters for items having multiple nonordered responses. Expressions for the estimates of the asymptotic variances of the item parameters and on overall chi-square goodness of fit test are also provided.

It is shown that the common and unique variance estimates produced by Martin & McDonald's Bayesian estimation procedure for the unrestricted common factor model have a predictable sum which is always greater than the maximum likelihood estimate of the total variance. This fact is used to justify a suggested simple alternative method of specifying the Bayesian parameters required by the procedure.

A goodness of fit test presented by Andersen is shown to be incorrect. The correct test is described and a re-analysis of Andersen's data is provided.