Search

Growth is a fundamental aspect of a living organism. Growth curves play an important role in explaining the complex dynamics of growth trajectories. The development of a large class of growth models provides more choices to explain complex growth dynamics. However, identifying a suitable growth curve from a broad class of growth models becomes a challenging task. Relative Growth Rate (RGR) is the most popular measure in the growth-related study. It serves many purposes in growth curve literature, including constructing any goodness-of-fit index of some growth dynamics. However, the goodness-of-fit test based on RGR is restricted to only simple growth models. This study aims to develop a new growth rate function, instantaneous maturity rate (IMR), which can play an important role in identifying growth models. We have explored that the measure has synergy in mathematical form with IMR. However, unlike the hazard rate, IMR is a random variable when the size/RGR variable is stochastic. We have derived the exact and asymptotic distribution of this measure under the Gaussian setup of both the size and RGR variables. We have constructed a goodness-of-fit test for the extended Gompertz growth model based on the instantaneous maturity rate. We have checked the performance of the test through simulation studies as well as real data. AMS 2010 subject classifications: 62Mxx, 92Bxx, 62P10

Factor mixture modeling (FMM) has been increasingly used to investigate unobserved population heterogeneity. This study examined the issue of covariate effects with FMM in the context of measurement invariance testing. Specifically, the impact of excluding and misspecifying covariate effects on measurement invariance testing and class enumeration was investigated via Monte Carlo simulations. Data were generated based on FMM models with (1) a zero covariate effect, (2) a covariate effect on the latent class variable, and (3) covariate effects on both the latent class variable and the factor. For each population model, different analysis models that excluded or misspecified covariate effects were fitted. Results highlighted the importance of including proper covariates in measurement invariance testing and evidenced the utility of a model comparison approach in searching for the correct specification of covariate effects and the level of measurement invariance. This approach was demonstrated using an empirical data set. Implications for methodological and applied research are discussed.

The purpose of this study was to develop a valid and reliable rubric to be used for the evaluation of large ensemble wind band performances. The guiding questions for this study were: (a) what are the psychometric qualities (i.e., reliability and validity) of the scale developed to assess wind band ensemble performance at the high school level? (b) how do the items fit the model and vary in difficulty? (c) how does the structure of the rating scale vary across individual items? and (d) how can the rating scale be transferred into an informative rubric? The primary data analysis tool used in this study was the Multifaceted Rasch Partial Credit Measurement Model. Music content experts ( N = 20) were solicited to evaluate 40 wind band performances, each evaluator listening to four. A 4-point Likert-type rating scale (e.g., Strongly Agree, Agree, Disagree, and Strongly Disagree) was used to evaluate each recorded performance. Results indicated good model data fit and resulted in a final rubric containing 24 items ranging from two to four performance categories. Implications for classroom teaching and consequential validity are discussed.

Ordinal regression models are used to describe the relationship between an ordered categorical response variable and one or more explanatory variables. Several ordinal logistic models are available in Stata, such as the proportional odds, adjacent-category, and constrained continuation-ratio models. In this article, we present a command (ologitgof) that calculates four goodness-of-fit tests for assessing the overall adequacy of these models. These tests include an ordinal version of the Hosmer–Lemeshow test, the Pulkstenis–Robinson chi-squared and deviance tests, and the Lipsitz likelihood-ratio test. Together, these tests can detect several different types of lack of fit, including wrongly specified continuous terms, omission of different types of interaction terms, and an unordered response variable.

Though rare, crashes at highway–railway crossings usually result in severe injuries and fatalities. Intersections along railway corridors share many conditions that support the use of spatial analyses, but so far the spatial effects are essentially unknown. Little research has been conducted about highway–railway crossings using spatial correlation. The study presented in this paper analyzed spatial correlation structures on crash frequency at railway crossings, particularly conditional autoregressive (CAR) and joint specification, with the use of Gaussian Kriging models. Full Bayesian Poisson–lognormal approaches were used to compare the effects of various models, including heterogeneity-only, spatial-only, and heterogeneity–spatial models. These methods were estimated with crash data from a low-speed passenger train service in Costa Rica. The deviance information criterion was used to compare the models. Heterogeneity–CAR models show a better goodness of fit than heterogeneity-only, joint Kriging, and CAR-only definitions. The second-order neighboring model (CAR-2) yielded the best fit and related to an average neighbor distance of about 700 m (0.44 mi). Robust semi-variograms based on joint Kriging models and CAR methods showed similar results. The proportion of variation in the data explained by spatial correlation methods was similar (29% of the total variation). These findings suggest that spatial correlation at highway–railway crossings should be considered in modeling of crash frequencies.

Modeling growth for students with significant cognitive disabilities (SWSCD) is difficult due to a variety of factors, including, but not limited to, missing data, test scaling, group heterogeneity, and small sample sizes. These challenges may account for the paucity of previous research exploring the academic growth of SWSCD. Our study represents a unique context in which a reading assessment, calibrated to a common scale, was administered statewide to students in consecutive years across Grades 3 to 5. We used a nonlinear latent growth curve pattern-mixture model to estimate students’ achievement and growth while accounting for patterns of missing data. While we observed significant intercept differences across disability subgroups, there were no significant slope differences. Incorporating missing data patterns into our models improved model fit. Limitations and directions for future research are discussed.

Fuzzy-set qualitative comparative analysis (fsQCA) has become one of the most prominent methods in the social sciences for capturing causal complexity, especially for scholars with small- and medium- N data sets. This research note explores two key assumptions in fsQCA’s methodology for testing for necessary and sufficient conditions—the cumulation assumption and the triangular data assumption—and argues that, in combination, they produce a form of aggregation bias that has not been recognized in the fsQCA literature. It also offers a straightforward test to help researchers answer the question of whether their findings are plausibly the result of aggregation bias.

This article introduces an entropy-based measure of data–model fit that can be used to assess the quality of logistic regression models. Entropy has previously been used in mixture-modeling to quantify how well individuals are classified into latent classes. The current study proposes the use of entropy for logistic regression models to quantify the quality of classification and separation of group membership. Entropy complements preexisting measures of data–model fit and provides unique information not contained in other measures. Hypothetical data scenarios, an applied example, and Monte Carlo simulation results are used to demonstrate the application of entropy in logistic regression. Entropy should be used in conjunction with other measures of data–model fit to assess how well logistic regression models classify cases into observed categories.

A latent variable modeling procedure is discussed that can be used to test if two or more homogeneous multicomponent instruments with distinct components are measuring the same underlying construct. The method is widely applicable in scale construction and development research and can also be of special interest in construct validation studies. The approach can be readily utilized in empirical settings with observed measure nonnormality and/or incomplete data sets. The procedure is based on testing model nesting restrictions, and it can be similarly employed to examine the collapsibility of latent variables evaluated by multidimensional measuring instruments. The outlined method is illustrated with two data examples.

Effects of rating scale forms on cross-sectional reliability and measurement equivalence were investigated. A randomized experimental design was implemented, varying category labels and number of categories. The participants were 800 students at two German universities. In contrast to previous research, reliability assessment method was used, which relies on the congeneric measurement model. The experimental manipulation had differential effects on the reliability scores and measurement equivalence. Attitude strength seems to be a relevant moderator variable, which influences measurement equivalence. Overall, the results show that measurement quality is influenced by rating scale forms. Results are discussed in terms of their implications for latent variables measurement.

The literature proposes numerous so-called pseudo-R2 measures for evaluating “goodness of fit” in regression models with categorical dependent variables. Unlike ordinary least square-R2, log-likelihood-based pseudo-R2s do not represent the proportion of explained variance but rather the improvement in model likelihood over a null model. The multitude of available pseudo-R2 measures and the absence of benchmarks often lead to confusing interpretations and unclear reporting. Drawing on a meta-analysis of 274 published logistic regression models as well as simulated data, this study investigates fundamental differences of distinct pseudo-R2 measures, focusing on their dependence on basic study design characteristics. Results indicate that almost all pseudo-R2s are influenced to some extent by sample size, number of predictor variables, and number of categories of the dependent variable and its distribution asymmetry. Hence, an interpretation by goodness-of-fit benchmark values must explicitly consider these characteristics. The authors derive a set of goodness-of-fit benchmark values with respect to ranges of sample size and distribution of observations for this measure. This study raises awareness of fundamental differences in characteristics of pseudo-R2s and the need for greater precision in reporting these measures.

The factor structure of the Teacher–Child Rating Scale (T-CRS 2.1) was examined using confirmatory factor analysis (CFA). A cross-sectional study was carried out on 68,497 children in prekindergarten through Grade 10. Item reduction was carried out based on modification indices, standardized residual covariance, and standardized factor loadings. A higher order model with a general super-ordinate factor fit the data well, and is consistent with the notion of a unidimensional non-cognitive set of learning-related skills. Model-based reliability estimates are provided.

Many studies have addressed spatial correlation in traffic collision modeling. It has been generally concluded that the inclusion of spatial correlation improves model goodness of fit and the precision of parameter estimates. However, the application in before–after safety evaluation has rarely been documented in the traffic safety literature. The objectives of the presented study were to ( a) apply both the univariate and multivariate full Bayesian (FB) spatial models in before–after safety evaluation and ( b) compare the results with those of nonspatial FB models. A reduction of the posted speed limit in urban residential neighborhoods in Edmonton, Alberta, Canada was used as a case study for the before–after safety evaluation. Yearly collision data and other neighborhood characteristics data were collected for a group of treated and reference neighborhoods to develop macroscopic models. The four models considered in this study were ( a) Poisson–lognormal, ( b) Poisson–lognormal with conditional autoregressive (CAR) distribution, ( c) multivariate Poisson–lognormal, and ( d) multivariate Poisson–lognormal with CAR distribution. The results showed that the multivariate Poisson–lognormal with CAR distribution model for collision severities outperformed the other three models according to the deviance information criteria. Parameter estimates showed slight differences across the models. However, for the current data set, the results of the before–after safety evaluation showed similar findings across the models. Estimated collision reductions were 13%, 24%, and 12% for total, severe, and property-damage-only collisions, respectively.

Although differences in goodness-of-fit indices (ΔGOFs) have been advocated for assessing measurement invariance, studies that advanced recommended differential cutoffs for adjudicating invariance actually utilized a very limited range of values representing the quality of indicator variables (i.e., magnitude of loadings). Because quality of measurement has been found to be relevant in the context of assessing data-model fit in single-group models, this study used simulation and population analysis methods to examine the extent to which quality of measurement affects ΔGOFs for tests of invariance in multiple group models. Results show that ΔMcDonald’s NCI is minimally affected by loading magnitude and sample size when testing invariance in the measurement model, while differences in comparative fit index varies widely when testing both measurement and structural variance as measurement quality changes, making it difficult to pinpoint a common value that suggests reasonable invariance.

The purpose of this study is to validate the widely adopted Teachers’ Sense of Efficacy Scale (TSES) for the East Asian context. The researchers seek to find out whether TSES holds validity and reliability and is appropriate for use to measure teacher efficacy in China, Korea, and Japan. 489 teachers from the three countries participated in the study. CFA models were used to test the factorial structures of TSES for both 24-item long and 12-item short forms, and three-factor models were tested for both forms. The results showed that three-factor models did not fit well for all three country groups for the long form. The short form models’ fit indices for all three countries were acceptable. The researchers proposed an 11-item three-factor model by removing one item from the original TSES short form and further tested the revised 11-item TSES for the three Asian groups. Results suggested that the 11-item model provided good model-data fit. In the following measurement invariance test, the 11-item TSES was used as the baseline model. The results from the test of the measurement invariance of the revised 11-item TSES scale suggested that this scale can be used in cross-culture studies about teacher efficacy in the East Asian context. The instrument may produce results that are possible to conduct comparisons across the three cultures.

Recent research in travel behavior has contributed numerous technical developments for the estimation of discrete choice models with latent attributes, including the hybrid choice model (HCM). However, assessment of goodness of fit, reliability, validity, and predictive capacities of the joint model remain open research questions. The HCM is a special form of structural equation modeling (SEM). Several goodness-of-fit indexes are all in standard use in psychometric SEM. In this paper, the validity of these indexes is examined for the HCM case. Behavior of SEM fit assessment tools is known in factor analysis (some controversies in this area are reviewed in this paper), but performance of these indexes in the HCM has not been studied. A Monte Carlo study, as well as empirical microdata on bicycle route choice, was used to show that standard SEM fit assessment did not work as expected for the HCM. Important differences were discovered in model fit between the HCM and the multiple indicator multiple cause (MIMIC) model with the same structural and measurement equations for the latent attributes. Sometimes the HCM was rejected when indexes failed to reject the MIMIC structure and vice versa. One of the sources of this divergence was that the measurement equation of the choice kernel did not have an error term; this assumption was nonstandard in SEM. Until a uniform method for measuring the HCM goodness of fit is found, it is recommended that the chi-square test be used for the MIMIC component of the joint model.

Latent variables are common in psychological research. Research questions involving the interaction of two variables are likewise quite common. Methods for estimating and interpreting interactions between latent variables within a structural equation modeling framework have recently become available. The latent moderated structural equations (LMS) method is one that is built into Mplus software. The potential utility of this method is limited by the fact that the models do not produce traditional model fit indices, standardized coefficients, or effect sizes for the latent interaction, which renders model fitting and interpretation of the latent variable interaction difficult. This article compiles state-of-the-science techniques for assessing LMS model fit, obtaining standardized coefficients, and determining the size of the latent interaction effect in order to create a tutorial for new users of LMS models. The recommended sequence of model estimation and interpretation is demonstrated via a substantive example and a Monte Carlo simulation. Finally, extensions of this method are discussed, such as estimating quadratic effects of latent factors and interactions between latent slope and intercept factors, which hold significant potential for testing and advancing developmental theories.

In item response theory (IRT), the underlying latent variables are typically assumed to be normally distributed. If the assumption of normality is violated, the item and person parameter estimates can become biased. Therefore, it is necessary in practical data analysis situations to examine the adequacy of this assumption in an effective manner. There is a recent surge of interest in limited-information overall goodness-of-fit test statistics for IRT models (see e.g., Cai, Maydeu-Olivares, Coffman, & Thissen, 2006; Joe & Maydeu-Olivares, 2010; Cai & Hansen, 2013), but their appropriateness for diagnosing latent variable distributional fit has not been studied. The approach undertaken in this research is to use summed score likelihood based indices. The idea itself is not new (see e.g., Ferrando & Lorenzo-Seva, 2001; Hambleton & Traub, 1973; Lord, 1953; Ross, 1966; Sinharay, Johnson, & Stern, 2006; Thissen & Wainer, 2001), but this study recasts the problem using the framework of limited-information goodness of fit testing. The summed score based indices can be viewed as a particular form of reduction of the full underlying multinomial that are potentially sensitive to the latent variable distributional misspecifications.Results from a pilot study (Li & Cai, 2012) show that summed score likelihood based indices enjoy high statistical power for detecting latent variable distributional assumption violations, and are not sensitive (correctly) to other forms of model misspecification such as unmodeled multidimensionality. Meanwhile, the limited-information overall fit statistic M2 (Maydeu-Olivares & Joe, 2005) has relatively low power against latent variable non-normality. However, technically the statistical indices proposed by Li and Cai (2012) don't follow an exactly chi-squared distribution. They proposed a heuristic degrees of freedom adjustment, but more rigorous justifications could be developed along the lines of Sattora-Bentler type moment adjustment popular in structural equation modeling (Satorra & Bentler, 1994). In IRT, the moment adjustment approaches have been used by Cai et al. (2006) and Maydeu-Olivares (2001). The major methodological contributions of my dissertation come from simulation studies that examine the calibration and power of the moment adjusted test statistics across various conditions: number of items, sample size, item type, generating latent variable distribution, and the values of generating item parameters. The performance of these fit statistics are also compared with M2. Simulation study results show that the proposed moment-adjusted statistics improve upon the unadjusted statistics in the null and alternative conditions, especially when generating item parameters are dispersed. Finally, performance of the indices is illustrated with empirical data from educational and psychological assessment development projects.

Researchers using factor analysis tend to dismiss the significant ill fit of factor models by presuming that if their factor model is close-to-fitting, it is probably close to being properly causally specified. Close fit may indeed result from a model being close to properly causally specified, but close-fitting factor models can also be seriously causally misspecified. This article illustrates a variety of nonfactor causal worlds that are perfectly, but inappropriately, fit by factor models. Seeing nonfactor worlds that are perfectly yet erroneously fit via factor models should help researchers understand that close-to-fitting factor models may seriously misrepresent the world’s causal structure. Statistical cautions regarding the factor model’s proclivity to fit when it ought not to fit have been insufficiently publicized and are rarely heeded. A research commitment to understanding the world’s causal structure, combined with clear examples of factor mismodeling should spur diagnostic assessment of significant factor model failures—including reassessment of published failing factor models.

The analysis of disease risk is often considered via relative risk. The comparison of relative risk estimation methods with “true risk” scenarios has been considered on various occasions. However, there has been little examination of how well competing methods perform when the focus is clustering of risk. In this paper, a simulated evaluation of a range of potential spatial risk models and a range of measures that can be used for (a) cluster goodness of fit, (b) cluster diagnostics are considered. Results suggest that exceedence probability is a poor measure of hot spot clustering because of model dependence, whereas residual-based methods are less model dependent and perform better. Local deviance information criteria measures perform well, but conditional predictive ordinate measures yield a high false positive rate.

Conventional differential item functioning (DIF) detection methods (e.g., the Mantel–Haenszel test) can be used to detect DIF only across observed groups, such as gender or ethnicity. However, research has found that DIF is not typically fully explained by an observed variable. True sources of DIF may include unobserved, latent variables, such as personality or response patterns. The factor mixture model (FMM) is designed to detect unobserved sources of heterogeneity in factor models. The current study investigated use of the FMM for detecting between-class latent DIF and class-specific observed DIF. Factors that were manipulated included the DIF effect size and the latent class probabilities. The performance of model fit indices (Akaike information criterion [AIC], Bayesian information criterion [BIC], sample size–adjusted BIC, and consistent AIC) were assessed for their detection of the correct DIF model. The recovery of DIF parameters was also assessed. Results indicated that use of FMMs with binary outcomes performed well in terms of the DIF detection and for recovery of large DIF effects. When class probabilities were unequal with small DIF effects, performance decreased for fit indices, power, and the recovery of DIF effects compared with equal class probability conditions. Inflated Type I errors were found for non-DIF items across simulation conditions. Results and future research directions for applied and methodological are discussed.

The present study assessed the impact of sample size on the power and fit of structural equation modeling applied to functional brain connectivity hypotheses. The data consisted of time-constrained minimum norm estimates of regional brain activity during performance of a reading task obtained with magnetoencephalography. Power analysis was first conducted for an autoregressive model with 5 latent variables (brain regions), each defined by 3 indicators (successive activity time bins). A series of simulations were then run by generating data from an existing pool of 51 typical readers (aged 7.5-12.5 years). Sample sizes ranged between 20 and 1,000 participants and for each sample size 1,000 replications were run. Results were evaluated using chi-square Type I errors, model convergence, mean RMSEA (root mean square error of approximation) values, confidence intervals of the RMSEA, structural path stability, and Δ-Fit index values. Results suggested that 70 to 80 participants were adequate to model relationships reflecting close to not so close fit as per MacCallum et al.’s recommendations. Sample sizes of 50 participants were associated with satisfactory fit. It is concluded that structural equation modeling is a viable methodology to model complex regional interdependencies in brain activation in pediatric populations.

Growth mixture modeling has gained much attention in applied and methodological social science research recently, but the selection of the number of latent classes for such models remains a challenging issue, especially when the assumption of proper model specification is violated. The current simulation study compared the performance of a linear growth mixture model (GMM) for determining the correct number of latent classes against a completely unrestricted multivariate normal mixture model. Results revealed that model convergence is a serious problem that has been underestimated by previous GMM studies. Based on two ways of dealing with model nonconvergence, the performance of the two types of mixture models and a number of model fit indices in class identification are examined and discussed. This article provides suggestions to practitioners who want to use GMM for their research.

Item response theory (IRT) models allow model–data fit to be assessed at the individual level by using person-fit indices. This assessment is also feasible when IRT is used to model test–retest data. However, person-fit developments for this type of modeling are virtually nonexistent. This article proposes a general person-fit approach for test–retest data, which is based on practical likelihood-based indices. The approach is intended for two types of assumption regarding trait levels—stability and change—and can be used with a variety of IRT models. It consists of two groups of indices: (a) overall indices based on the full test–retest pattern, which are more powerful and are intended to flag a respondent as potentially inconsistent; and (b) partial indices intended to provide additional information about the location and sources of misfit. Furthermore, because the overall procedures assume local independence under repetition, a statistic for assessing the presence of retest effects at the individual level is also proposed. The functioning of the procedures was assessed by using simulation and is illustrated with two empirical studies: a stability study based on graded-response items and a change study based on binary items. Finally, limitations and further lines of research are discussed.

In the social sciences, latent traits often have a hierarchical structure, and data can be sampled from multiple levels. Both hierarchical latent traits and multilevel data can occur simultaneously. In this study, we developed a general class of item response theory models to accommodate both hierarchical latent traits and multilevel data. The freeware WinBUGS was used for parameter estimation. A series of simulations were conducted to evaluate the parameter recovery and the consequence of ignoring the multilevel structure. The results indicated that the parameters were recovered fairly well; ignoring multilevel structures led to poor parameter estimation, overestimation of test reliability for the second-order latent trait, and underestimation of test reliability for the first-order latent traits. The Bayesian deviance information criterion and posterior predictive model checking were helpful for model comparison and model-data fit assessment. Two empirical examples that involve an ability test and a teaching effectiveness assessment are provided.

Item response theory (IRT) is a psychometric technique used in the development, evaluation, improvement, and scoring of multi-item scales. This pedagogical article provides the necessary information needed to understand how to conduct, interpret, and report results from two commonly used ordered polytomous IRT models (Samejima’s graded response [GR] model and reduced GR model). Throughout this article, simulated data from a multi-item scale is used to illustrate IRT analyses. The simulated data and IRTPRO version 2.1 point-and-click commands needed to reproduce all analyses in this article are available as supplemental online materials at http://jea.sagepub.com/maint . The intent of this article is to provide an overview of essential components of an IRT analysis to enable increased access to this powerful tool for applied early adolescence researchers.

In this study, we reanalyzed the classic Hawai’i Family Study of Cognition (HFSC) data using contemporary multilevel modeling techniques. We used the HFSC baseline data ( N = 6,579) and reexamined the factorial structure of 16 cognitive variables using confirmatory (restricted) measurement models in an explicit sequence. These models were initially fitted using multilevel confirmatory factor analysis techniques and the invariant six-factor models with two higher order factors fit fairly well (εa < 0.08) to the total, between- and within-family data. More crucially, a model requiring metric factorial invariance proved to be a reasonable fit to the between and within matrices, and allowed the ratio of the between-family variation to total family variation (eta-squared) to be calculated separately for each common factor (η2: Gc = .27, Gf = .22, Gm = .15, Gs = .04, Gv = .30, and SP = .16). Higher order factors were fitted using multilevel structural equation modeling techniques and these suggested a reasonable two-factor solution with unequal family impacts. These results suggest that (a) A “G only model” does not fit the data very well, and there are many sources of individual differences in cognitive abilities; (b) the sources of the individual differences in cognition can be measured the same way between and within families; and (c) even after the unique test components are removed, cognitive differences are larger within families than between families. We consider other general multivariate family models, and we raise questions about family influences.

Longitudinal and clustered data, where multiple observations for individuals are observed, require special models that reflect their hierarchical structure. The most commonly used such model is the linear multilevel model, which combines a linear model for the population-level fixed effects, a linear model for normally distributed individual-level random effects and normally distributed observation-level errors with constant variance. It has the advantage of simplicity of interpretation, but if the assumptions of the model do not hold inferences drawn can be misleading. In this paper, we discuss the use of regression trees that are designed for multilevel data to construct goodness-of-fit tests for this model that can be used to test for nonlinearity of the fixed effects or heteroscedasticity of the errors. Simulations show that the resultant tests are slightly conservative as 0.05 level tests, and have good power to identify explainable model violations (that is, ones that are related to available covariate information in the data). Application of the tests is illustrated on two real datasets.

Newer statistical procedures are typically introduced to help address the limitations of those already in practice or to deal with emerging research needs. Quantile regression (QR) is introduced in this paper as a relatively new methodology, which is intended to overcome some of the limitations of least squares mean regression (LMR). QR is more appropriate when assumptions of normality and homoscedasticity are violated. Also QR has been recommended as a good alternative when the research literature suggests that explorations of the relationship between variables need to move from a focus on average performance, that is, the central tendency, to exploring various locations along the entire distribution. Although QR has long been used in other fields, it has only recently gained popularity in educational statistics. For example, in the ongoing push for accountability and the need to document student improvement, the calculation of student growth percentiles (SGP) utilizes QR to document the amount of growth a student has made. Despite its proven advantages and its utility, QR has not been utilized in areas such as language testing research. This paper seeks to introduce the field to basic QR concepts, procedures, and interpretations. Researchers familiar with LMR will find the comparisons made between the two methodologies helpful to anchor the new information. Finally, an application with real data is employed to demonstrate the various analyses (the code is also appended) and to explicate the interpretations of results.

In the field of international educational surveys, equivalence of achievement scale scores across countries has received substantial attention in the academic literature; however, only a relatively recent emphasis on scale score equivalence in nonachievement education surveys has emerged. Given the current state of research in multiple-group models, findings regarding these recent measurement invariance investigations were supported with research that was limited in scope to few groups and relatively small sample sizes. To that end, this study uses data from one large-scale survey as a basis for examining the extent to which typical fit measures used in multiple-group confirmatory factor analysis are suitable for detecting measurement invariance in a large-scale survey context. Using measures validated in a smaller scale context and an empirically grounded simulation study, our findings indicate that many typical measures and associated criteria are either unsuitable in a large group and varied sample-size context or should be adjusted, particularly when the number of groups is large. We provide specific recommendations and discuss further areas for research.

Using a multilevel latent growth curve modeling (LGCM) approach, this study examined longitudinal change in levels of physical fitness performance over time (i.e. four years) in young adolescents aged from 12–13 years. The sample consisted of 6622 students from 138 secondary schools in Singapore. Initial analyses found between-school variation on fitness test scores with intra-class correlations ranging from 0.02–0.19. Subsequent multilevel growth curve analyses revealed a quadratic trend of the longitudinal data across five stations of performance tests, with significant within-school student variability in change over time. The result of the multilevel LGCM showed that there were strong school effects on all of the physical fitness performances, in addition to inter- and intra-individual differences.

A significant aspect of data modeling with categorical predictors is the definition of a saturated model. In fact, there are different ways of specifying it—the casewise, the contingency table, and the collapsing approaches—and they strictly depend on the unit of analysis considered. The analytical units of reference could be the subjects or, alternatively, groups of subjects that have the same covariate pattern. In the first case, the goal is to predict the probability of success (failure) for each individual; in the second case, the goal is to predict the proportion of successes (failures) in each group. The analytical unit adopted does not affect the estimation process; however, it does affect the definition of a saturated model. Consequently, measures and tests of goodness of fit can lead to different results and interpretations. Thus one must carefully consider which approach to choose. In this article, we focus on the deviance test for logistic regression models. However, the results and the conclusions are easily applicable to other linear models involving categorical regressors. We show how Stata 12.1 performs when implementing goodness of fit. In this situation, it is important to clarify which one of the three approaches is implemented as default. Furthermore, a prominent role is played by the shape of the dataset considered (individual format or events–trials format) in accordance with the analytical unit choice. In fact, the same procedure applied to different data structures leads to different approaches to a saturated model. Thus one must attend to practical and theoretical statistical issues to avoid inappropriate analyses.

The relationship between saturated path-analysis models and their fit to data is revisited. It is demonstrated that a saturated model need not fit perfectly or even well a given data set when fit to the raw data is examined, a criterion currently frequently overlooked by researchers utilizing path analysis modeling techniques. The potential of individual case residuals for saturated model fit assessment is revealed by showing how they can be used to examine local fit, as opposed to overall fit, to sense possible model deficiencies or misspecifications, and to suggest model improvements when needed. The discussion is illustrated with several numerical examples.

Testing goodness of fit is an important step in evaluating a statistical model. For binary logistic regression models, the Hosmer–Lemeshow goodness-of-fit test is often used. For multinomial logistic regression models, however, few tests are available. We present the mlogitgof command, which implements a goodness-of-fit test for multinomial logistic regression models. This test can also be used for binary logistic regression models, where it gives results identical to the Hosmer–Lemeshow test.

Both testlet design and hierarchical latent traits are fairly common in educational and psychological measurements. This study aimed to develop a new class of higher order testlet response models that consider both local item dependence within testlets and a hierarchy of latent traits. Due to high dimensionality, the authors adopted the Bayesian approach implemented in the WinBUGS freeware for parameter estimation. A series of simulations were conducted to evaluate parameter recovery, consequences of model misspecification, and effectiveness of model–data fit statistics. Results show that the parameters of the new models can be recovered well. Ignoring the testlet effect led to a biased estimation of item parameters, underestimation of factor loadings, and overestimation of test reliability for the first-order latent traits. The Bayesian deviance information criterion and the posterior predictive model checking were helpful for model comparison and model–data fit assessment. Two empirical examples of ability tests and nonability tests are given.

Models of modern test theory imply statistical independence among responses, generally referred to as local independence. One violation of local independence occurs when the response to one item governs the response to a subsequent item. Expanding on a formulation of this kind of violation as a process in the dichotomous Rasch model, this article generalizes the dependence process to the case of the unidimensional, polytomous Rasch model. It then shows how the magnitude of this violation can be estimated as a change in the location of thresholds separating adjacent categories in the second item caused by the response dependence on the first. As in the dichotomous model, it is suggested that this index is relatively more tangible in interpretation than other indices of dependence that are either a weight in the interaction term in a model or a correlation coefficient. One function of this method of assessing dependence is likely to be in the development of tests and assessment formats where evidence of the magnitude of dependence of one item on another in a pilot study can be used as part of the evidence in deciding which items will be retained in a final version of a test or which formats might need to be reconstructed. A second function might be to identify the magnitude of response dependence that may then need to be taken into account in some other way, perhaps by applying a model that takes account of the dependence.

Two Monte Carlo simulation studies investigated the effectiveness of the mean adjusted χ2/ df statistic proposed by Drasgow and colleagues and, because of problems with the method, a new approach for assessing the goodness of fit of an item response theory model was developed. It has been previously recommended that mean adjusted χ2/ df values greater than 3 using a cross-validation data set indicate substantial misfit. The authors used simulations to examine this critical value across different test lengths (15, 30, 45) and sample sizes (500, 1,000, 1,500, 5,000). The one-, two- and three-parameter logistic models were fitted to data simulated from different logistic models, including unidimensional and multidimensional models. In general, a fixed cutoff value was insufficient to ascertain item response theory model–data fit. Consequently, the authors propose the use of the parametric bootstrap to investigate misfit and evaluated its performance. This new approach produced appropriate Type I error rates and had substantial power to detect misfit across simulated conditions. In a third study, the authors applied the parametric bootstrap approach to LSAT data to determine which dichomotous item response theory model produced the best fit. Future applications of the mean adjusted χ2/ df statistic are discussed.

Researchers commonly compare means and other statistics across groups with little concern for whether the measure possesses strong factorial invariance (i.e., equal factor loadings and intercepts/thresholds). When this assumption is violated, inaccurate inferences associated with statistical and practical significance can occur. This manuscript emphasizes the importance of testing for measurement invariance (MI) and provides guidance when conducting these tests. Topics discussed are potential causes of noninvariant items, the difference between measurement bias and invariance, remedies for noninvariant measures, and considerations associated with model estimation. Using a sample of 491 teachers, a demonstration is also provided that evaluates whether a newly constructed behavior and instructional management scale is invariant across elementary and middle school teachers. Analyses revealed that the results differ slightly based on the estimation method utilized although these differences did not greatly influence the latent factor mean difference conclusions. Additional implications and considerations related to invariance testing are discussed.

The most popular measures of multidimensional constructs typically fail to meet standards of good measurement: goodness of fit, measurement invariance, lack of differential item functioning, and well-differentiated factors that are not so highly correlated as to detract from their discriminant validity. Part of the problem, the authors argue, is undue reliance on overly restrictive independent cluster models of confirmatory factor analysis (ICM-CFA) in which each item loads on one, and only one, factor. Here the authors demonstrate exploratory structural equation modeling (ESEM), an integration of the best aspects of CFA and traditional exploratory factor analyses (EFA). On the basis of responses to the 11-factor Motivation and Engagement Scale ( n = 7,420, Mage = 14.22), we demonstrate that ESEM fits the data much better and results in substantially more differentiated (less correlated) factors than corresponding CFA models. Guided by a 13-model taxonomy of ESEM full-measurement (mean structure) invariance, the authors then demonstrate invariance of factor loadings, item intercepts, item uniquenesses, and factor variancescovariances, across gender and over time. ESEM has broad applicability to other areas of research that cannot be appropriately addressed with either traditional EFA or CFA and should become a standard tool for use in psychometric tests of psychological assessment instruments.

To realize the benefits of item response theory (IRT), one must have model-data fit. One facet of a model-data fit investigation involves assessing the tenability of the conditional item independence (CII) assumption. In this Monte Carlo study, the comparative performance of 10 indices for identifying conditional item dependence is assessed. The relative performances of these indices are evaluated in terms of their Type I error rate, power, and false positive rate. Results show that the residual correlations approach and the Q3 statistic can be considered as offering a reasonable compromise between maximum power and minimized false positive rates.

Using a Monte Carlo experimental design, this research examined the relationship between answer patterns’ aberrance rates and person-fit statistics (PFS) accuracy. It was observed that as the aberrance rate increased, the detection rates of PFS also increased until, in some situations, a peak was reached and then the detection rates of PFS decreased with increases in aberrance rates. Furthermore, the results suggest that ECI2Z was somewhat more robust to high levels of aberrance than lz, HT, and U3 when cheating was simulated. The results of this study shed light on a limitation of PFS analysis.

A two-step process is commonly used to evaluate data–model fit of latent variable path models, the first step addressing the measurement portion of the model and the second addressing the structural portion of the model. Unfortunately, even if the fit of the measurement portion of the model is perfect, the ability to assess the fit within the structural portion is affected by the quality of the factor–variable relations within the measurement model. The result is that models with poorer quality measurement appear to have better data–model fit, whereas models with better quality measurement appear to have worse data–model fit. The current article illustrates this phenomenon across different classes of fit indices, discusses related structural assessment problems resulting from issues of measurement quality, and endorses a supplemental modeling step evaluating the structural portion of the model in isolation from the measurement model.

The distance between nonparametric and parametric item characteristic curves has been proposed as an index of goodness of fit in item response theory in the form of a root integrated squared error index. This article proposes to use the posterior distribution of the latent trait as the nonparametric model and compares the performance of an index based on this method with another approach based on the kernel-smoothing model. Error rates and power are evaluated using the two-parameter logistic model and three types of realistic misfitted items. Results show that for fitting items, the distance between parametric and nonparametric item characteristic curves decreased as the sample size increased for both procedures. Kernel-smoothing root integrated squared error also decreased as test length increased. Bootstrap methods are used to obtain a significance test. Both procedures performed adequately in terms of Type I error rates. Regarding power, the posterior probabilities method was superior, especially in small samples, although in short tests both procedures performed in a similar way.

This study investigated the relative model—data fit of an ideal point item response theory (IRT) model (the generalized graded unfolding model [GGUM]) and dominance IRT models (e.g., the two-parameter logistic model [2PLM] and Samejima’s graded response model [GRM]) to simulated dichotomous and polytomous data generated from each of these models. The relative magnitudes of the adjusted χ2/ df ratios for item pairs and item triples at the test level were used to evaluate fit. Two simulation studies were conducted, one for dichotomous data and the other for polytomous data. Relative fit of the ideal point and dominance models were compared with respect to different conditions: test length, sample size, and sample type. In many simulated conditions, it was found that comparing relative fits (using test-level doubles and triples adjusted χ2/ df ratios) almost always consistently pointed to the correct IRT model. However, GGUM could fit dichotomous two-parameter logistic (2PL) data well when the scale length was short (i.e., 15 items); nevertheless, an examination of estimated GGUM item parameters clearly shows dominance item characteristics. Results of the simulation studies and implications are discussed.

Large variability and correlations among the coefficients obtained from the method of geographically weighted regression (GWR) have been identified in previous research. This is an issue that poses a serious challenge for the utility of the method as a tool to investigate multivariate relationships. The objectives of this paper are to assess: (1) the ability of GWR to discriminate between a spatially constant processes and one with spatially varying relationships; and (2) to accurately retrieve spatially varying relationships. Extensive numerical experiments are used to investigate situations where the underlying process is stationary and nonstationary, and to assess the degree to which spurious intercoefficient correlations are introduced. Two different implementations of GWR and cross-validation approaches are assessed. Results suggest that judicious application of GWR can be used to discern whether the underlying process is nonstationary. Furthermore, evidence of spurious correlations indicates that caution must be exercised when drawing conclusions regarding spatial relationships retrieved using this approach, particularly when working with small samples. Keywords: GWR, correlation, locally linear estimation, simulation, goodness of fit, inference

Microsimulation models of land use characterize attributes of a household and its location, referred to as microdata in this study. However, methods for evaluating the goodness of fit between estimated and observed sets of agent-based microdata have not been investigated extensively. Although the attributes of a household include various items, such as the relationship with the household head and ages of the members, housing type and spatial location, number of cars owned, and income, the attributes can be classified into general categories. The objective of the present study is to develop a goodness-of-fit evaluation method for agent-based household microdata sets composed of generalized attributes. First, a distance measure between the estimated and observed microdata for each household is defined. In this definition a generalized scheme is introduced, whereby attributes are structured by the household composition, attributes of the member, and attributes of the household as a whole. The goodness of fit is measured on the basis of the minimum sum of distances for all households in the study area. The calculation cannot be carried out with just a conventional algorithm for microdata of a typical size because the number of calculations increases in proportion to the factorial (N!) of the number (N) of agents. Therefore, a genetic algorithm, especially one using symbiotic evolution, is developed to solve the problem. The effectiveness of the method in regard to accuracy and calculation feasibility is confirmed by using person trip survey data for the Sapporo metropolitan area in Japan.

We review a few popular statistical models for correlated binary outcomes, present maximum likelihood estimates for the model parameters, and discuss model selection issues using a variety of goodness-of-fit test statistics. We apply bootstrap strategies that are computationally efficient to evaluate the performance of goodness-of-fit statistics and observe that generally the power and the type I error rate of the goodness-of-fit statistics depend on the model under investigation. Our simulation results show that careful choice of goodness-of-fit statistics is an important issue especially when we have a small sample and the outcomes are highly correlated. Two biomedical applications are included.

Confirmatory factor analytic (CFA) models are frequently used in many areas of organizational research. Due to their popularity, CFA models and issues about their fit have received a vast amount of attention during the past several decades. The purpose of this study was to examine several measures of fit and the appropriateness of previously developed ‘‘rules of thumb’’ for their interpretation. First, an empirical example is used to illustrate the effects of nonnormality on maximum likelihood (ML) estimation and to demonstrate the importance of diagonally weighted least squares (DWLS) estimation for organizational research. Then, the results of a simulation study are presented to show that appropriate cutoff values for DWLS estimation vary considerably across conditions. Finally, regression equations are described to aid researchers in selecting cutoff values for assessing the fit of DWLS solutions, given a desired level of Type I error. The results summarized here have important implications for the interpretation and use of CFA models.

In this article the authors develop goodness-of-fit tests for fuzzy-set analyses to formally assess the fit between empirical information and various causal hypotheses while accounting for measurement error in membership scores. These goodness-of-fit tests, and the accompanying logic, provide a sound inferential foundation for fuzzy-set methodology. The authors also develop descriptive measures to complement these tests. Examples from Stryker and Eliason (2003) and Mahoney (2003) show how goodness-of-fit tests and descriptive measures may be used to assess individual causal factors as well as conjunctions of factors. The authors show how these tools provide more information in a fuzzy-set analysis than do tests currently in use. In providing this inferential foundation, the authors also show that fuzzy-set methods (a) are no less amenable to falsificationist methods of the Neyman-Pearson type than are standard statistical techniques and (b) may be usefully applied in either an exploratory/inductive or a confirmatory/deductive research design.

The investigation of change in factor structure over time can provide new opportunities for the development of theory in psychology. The method proposed to investigate change in intraindividual factor structure over time is an extension of P-technique factor analysis, in which the P-technique factor model is fit within relatively small windows of time, followed by modeling the factor model parameter trajectories. This article describes a Monte Carlo study undertaken to evaluate the proposed estimation procedure and approach to model fit. An illustrative example is provided to motivate the proposed model, and directions for future research are outlined.