Search

Factor mixture modeling (FMM) has been increasingly used to investigate unobserved population heterogeneity. This study examined the issue of covariate effects with FMM in the context of measurement invariance testing. Specifically, the impact of excluding and misspecifying covariate effects on measurement invariance testing and class enumeration was investigated via Monte Carlo simulations. Data were generated based on FMM models with (1) a zero covariate effect, (2) a covariate effect on the latent class variable, and (3) covariate effects on both the latent class variable and the factor. For each population model, different analysis models that excluded or misspecified covariate effects were fitted. Results highlighted the importance of including proper covariates in measurement invariance testing and evidenced the utility of a model comparison approach in searching for the correct specification of covariate effects and the level of measurement invariance. This approach was demonstrated using an empirical data set. Implications for methodological and applied research are discussed.

This study examined the impact of omitting covariates interaction effect on parameter estimates in multilevel multiple-indicator multiple-cause models as well as the sensitivity of fit indices to model misspecification when the between-level, within-level, or cross-level interaction effect was left out in the models. The parameter estimates produced in the correct and the misspecified models were compared under varying conditions of cluster number, cluster size, intraclass correlation, and the magnitude of the interaction effect in the population model. Results showed that the two main effects were overestimated by approximately half of the size of the interaction effect, and the between-level factor mean was underestimated. None of comparative fit index, Tucker–Lewis index, root mean square error of approximation, and standardized root mean square residual was sensitive to the omission of the interaction effect. The sensitivity of information criteria varied depending majorly on the magnitude of the omitted interaction, as well as the location of the interaction (i.e., at the between level, within level, or cross level). Implications and recommendations based on the findings were discussed.

In multilevel multiple-indicator multiple-cause (MIMIC) models, covariates can interact at the within level, at the between level, or across levels. This study examines the performance of multilevel MIMIC models in estimating and detecting the interaction effect of two covariates through a simulation and provides an empirical demonstration of modeling the interaction in multilevel MIMIC models. The design factors include the location of the interaction effect (i.e., between, within, or across levels), cluster number, cluster size, intraclass correlation (ICC) level, magnitude of the interaction effect, and cross-level measurement invariance status. Type I error, power, relative bias, and root mean square of error of the interaction effects are examined. The results showed that multilevel MIMIC models performed well in detecting the interaction effect at the within or across levels. However, when the interaction effect was at the between level, the performance of multilevel MIMIC models depended on the magnitude of the interaction effect, ICC, and sample size, especially cluster number. Overall, cross-level measurement noninvariance did not make a notable impact on the estimation of interaction in the structural part of multilevel MIMIC models when factor loadings were allowed to be different across levels.

Wording effects associated with positively and negatively worded items have been found in many scales. Such effects may threaten construct validity and introduce systematic bias in the interpretation of results. A variety of models have been applied to address wording effects, such as the correlated uniqueness model and the correlated traits and correlated methods model. This study presents the multilevel bifactor approach to handling wording effects of mixed-format scales used in a multilevel context. The Students Confident in Mathematics scale is used to illustrate this approach. Results from comparing a series of models showed that positive and negative wording effects were present at both the within and the between levels. When the wording effects were ignored, the within-level predictive validity of the Students Confident in Mathematics scale was close to that under the multilevel bifactor model. However, at the between level, a lower validity coefficient was observed when ignoring the wording effects. Implications for applied researchers are discussed.

Monte Carlo methods were used to examine techniques for constructing confidence intervals around multivariate effect sizes. Using interval inversion and bootstrapping methods, confidence intervals were constructed around the standard estimate of Mahalanobis distance ( D2), two bias-adjusted estimates of D2, and Huberty’s I. Interval coverage and width were examined across conditions by adjusting sample size, number of variables, population effect size, population distribution shape, and the covariance structure. The accuracy and precision of the intervals varied considerably across methods and conditions; however, the interval inversion approach appears to be promising for D2, whereas the percentile bootstrap approach is recommended for the other effect size measures. The results imply that it is possible to obtain fairly accurate coverage estimates for multivariate effect sizes. However, interval width estimates tended to be large and uninformative, suggesting that future efforts might focus on investigating design factors that facilitate more precise estimates of multivariate effect sizes.

Three analytic approaches were employed within the framework of classical test theory to examine construct validity of a mathematics assessment made up of 16 ordered constructed-response tasks. The tasks focused on problem solving by identifying and explaining mathematical patterns. Descriptive analyses across four age groups suggested a developmental structure of individual tasks and subdomains that was generally consistent with the intended design of the test. Acceptable fit was obtained using structural equation modeling procedures for a five-factor, first-order structure with directional paths specified (RMSEA = .068) and for two second-order factors (RMSEA = .068). Convergent validity coefficients with standardized tests in mathematics computation, concepts of numbers, and applications indicated a modest overlap in constructs across the two assessments (r = .47) with Stanford Achievement Test (SAT) total mathematics scores. Findings are discussed with reference to intended uses of the assessment and its scores in the context of current mathematics education reforms.

The aim of this study was to investigate the factorial invariance of the Wechsler Intelligence Scale for Children-Third Edition (WISC-III) when administered to deaf children who use sign language. A multisample confirmatory factor analysis, using LISREL7, was used to test the four-factor model on the covariance matrices of subtest scale scores for deaf and standardization samples. The general form of the four-factor model fit for both samples. Path coefficients, error variances, and factor covariances were not invariant, indicating that test constructs may be differentially indicated across groups.

This study explores the dimensional structure of the Home Observation for the Measurement of the Environment-Short Form (HOME-SF). Confirmatory factor analysis was used to evaluate three hypothesized factor structures. Data pertain to 1,250 children up to 2 years old, assessed by the National Longitudinal Survey of Youth (NLSY). All models were found to be unsatisfactory. The global fit indexes were low, solutions were inadmissible, and many of the estimated factor loadings were small. This study indicates that the features of the home environment cannot be thought of as composing a single dimension, nor can they be thought of as composing dimensions corresponding to the NLSY scales or the original HOME scales. Hence further research is warranted to determine more precisely the character of what the HOME-SF measures.

Scores from the Teacher Readiness for Educational Reforms (TRER)instrument were validated using a six-phase, iterative model. Initial conceptualization, content validation, and pilot testing yielded a 61-item instrument with seven subdomains (Phases 1-4). Exploratory work (Phase 5) using principal axis factor extraction supported a five-factor structure (Data Set 1, n = 393). Further exploration with a five-factor free-path model and a more constrained structural model yielded satisfactory fit values (Bentler’s comparative fit indexes of .94 and .93, respectively). Deletion or collapsing of items in Phase 5 yielded a refined TRER with 43 items. Confirmatory work (Phase 6) with a new data set ( n = 392) showed little slippage in fit. Cronbach’s alpha values ranged from .78 to .96 in final subdomain scores.