Search

Within the frequentist structural equation modeling (SEM) framework, adjudicating model quality through measures of fit has been an active area of methodological research. Complicating this conversation is research revealing that a higher quality measurement portion of a SEM can result in poorer estimates of overall model fit than lower quality measurement models, given the same structural misspecifications. Through population analysis and Monte Carlo simulation, we extend the earlier research to recently developed Bayesian SEM measures of fit to evaluate whether these indices are susceptible to the same reliability paradox, in the context of using both uninformative and informative priors. Our results show that the reliability paradox occurs for RMSEA, and to some extent, gamma-hat and PPP (measures of absolute fit); but not CFI or TLI (measures of relative fit), across Bayesian (MCMC) and frequentist (maximum likelihood) SEM frameworks alike. Taken together, these findings indicate that the behavior of these newly adapted Bayesian fit indices map closely to their frequentist analogs. Implications for their utility in identifying incorrectly specified models are discussed.

Oblique bifactor models, where group factors are allowed to correlate with one another, are commonly used. However, the lack of research on the statistical properties of oblique bifactor models renders the statistical validity of empirical findings questionable. Therefore, the present study took the first step to examine the statistical properties of oblique bifactor models through Monte Carlo simulations. Study 1 showed that the classic oblique bifactor "measurement" models had severe convergence issues in many conditions. Even for converged replications, both factor loading and group factor correlation estimates were severely biased. Study 2 further showed that the classic oblique bifactor "predictive" models still had serious convergence problems, and structural parameters suffered from issues of severe estimation bias and low power. Fortunately, the augmentation approach, where one or multiple indicators are specified to load onto only the general factor but not any of the group factors, was useful in ameliorating these issues in both oblique bifactor measurement and predictive models. Tentative recommendations regarding the selection between oblique and orthogonal bifactor models and the approaches to finding augmenting indicators were provided.

The logistic and confined exponential curves are frequently used in studies of growth and learning. These models, which are nonlinear in their parameters, can be estimated using structural equation modeling software. This paper proposes a single combined model, a weighted combination of both models. Mplus, Proc Calis, and lavaan code for the model are provided. Monte Carlo simulations varying the number of measurement occasions (5, 10, and 15), internal consistency ([alpha] = 0.5, 0.7, and 0.8), and sample size (N = 1,000, 500, and 300) were examined to understand whether the model can be successfully fit with SEM software. Convergence failures were appreciable when model parameters were equal to special cases of logistic or confined exponential curves. At least ten measurement occasions and a moderate degree of reliability ([alpha] > 0.7) were required to identify the model as superior to its stand-alone alternatives.

Latent class analysis (LCA) is a widely used technique for detecting unobserved population heterogeneity in cross-sectional data. Despite its popularity, the performance of LCA is not well understood. In this study, we evaluate the performance of LCA with binary data by examining classification accuracy, parameter estimation accuracy, and coverage rates of confidence intervals (CIs) through Monte Carlo simulation studies. We address the issue of label switching with a distance-based relabeling approach and introduce an index to measure separation among latent classes. Our results show that classification accuracy, parameter estimation accuracy, and CI coverage rates are primarily influenced by class separation and the number of indicators used for LCA. We recommend using a large sample size to mitigate the effects of tiny class sizes. Additionally, the study finds that the parametric bootstrap CIs perform comparably well or better when compared with the CIs based on the standard maximum likelihood method.

We consider the problem of bias arising from conditioning on a post-outcome collider. We illustrate this with reference to Elwert and Winship (2014) but we go beyond their study to investigate the extent to which inverse probability weighting might offer solutions. We use linear models to derive expressions for the bias arising in different kinds of post-outcome confounding, and we show the specific situations in which inverse probability weighting will allow us to obtain estimates that are consistent or, if not consistent, less biased than those obtained via ordinary least squares regression.

In the current study, we compare propensity score (PS) matching methods for data with a cross-classified structure, where each individual is clustered within more than one group, but the groups are not hierarchically organized. Through a Monte Carlo simulation study, we compared sequential cluster matching (SCM), preferential within cluster matching (PWCM), greedy matching (GM), and optimal full matching (OFM), using propensity scores from four different models. The results indicated that the four matching methods performed well when PSs were estimated with logistic regression containing both level-1 and level-2 covariates. When the level-2 covariates were omitted in the logistic regression PS model, matching methods resulted in biased treatment effect estimates. However, omission of level-2 covariates did not result in biased estimates when the PS model was a logistic cross-classified random effects model (CCREM). SCM and PWCM outperformed GM and OFM with a logistic CCREM that included level-1 and level-2 covariates.

Purpose: Many studies using machine learning (ML) in speech, language, and hearing sciences rely upon cross-validations with single data splitting. This study's first purpose is to provide quantitative evidence that would incentivize researchers to instead use the more robust data splitting method of nested k-fold cross-validation. The second purpose is to present methods and MATLAB code to perform power analysis for ML-based analysis during the design of a study. Method: First, the significant impact of different cross-validations on ML outcomes was demonstrated using real-world clinical data. Then, Monte Carlo simulations were used to quantify the interactions among the employed cross-validation method, the discriminative power of features, the dimensionality of the feature space, the dimensionality of the model, and the sample size. Four different cross-validation methods (single holdout, 10-fold, train-validation-test, and nested 10-fold) were compared based on the statistical power and confidence of the resulting ML models. Distributions of the null and alternative hypotheses were used to determine the minimum required sample size for obtaining a statistically significant outcome (5% significance) with 80% power. Statistical confidence of the model was defined as the probability of correct features being selected for inclusion in the final model. Results: ML models generated based on the single holdout method had very low statistical power and confidence, leading to overestimation of classification accuracy. Conversely, the nested 10-fold cross-validation method resulted in the highest statistical confidence and power while also providing an unbiased estimate of accuracy. The required sample size using the single holdout method could be 50% higher than what would be needed if nested k-fold cross-validation were used. Statistical confidence in the model based on nested k-fold cross-validation was as much as four times higher than the confidence obtained with the single holdout-based model. A computational model, MATLAB code, and lookup tables are provided to assist researchers with estimating the minimum sample size needed during study design. Conclusion: The adoption of nested k-fold cross-validation is critical for unbiased and robust ML studies in the speech, language, and hearing sciences.

Although educational research and evaluation generally occur in multilevel settings, many analyses ignore cluster effects. Neglecting the nature of data from educational settings, especially in non-randomized experiments, can result in biased estimates with long-term consequences. Our manuscript improves the availability and understanding of artificial neural networks, an underutilized method trending in other disciplines. This method also shows promise for dealing with challenges faced by educational researchers, such as analyzing clustered data. Therefore, we simulated data to generalize the potential benefits of artificial neural networks to different data types. We also compared artificial neural networks to more familiar methods and investigated the time it demanded to perform each technique. Hence, readers can decide when it may be more appropriate to use one method instead of another.

Most researchers investigate the corrected item-total correlation of items when analyzing item discrimination in multi-dimensional structures under the Classical Test Theory, which might lead to underestimating item discrimination, thereby removing items from the test. Researchers might investigate the corrected item-total correlation with the factors to which that item belongs; however, getting a general overview of the entire test is impossible. Based on this problem, this study aims to recommend a new index to investigate item discrimination in two-dimensional structures through a Monte Carlo simulation. The new item discrimination index is evaluated by identifying sample size, item discrimination value, inter-factor correlation, and the number of categories. Based upon the results of the study it can be claimed that the proposed item discrimination index proves acceptable performance for two-dimensional structures. Accordingly, using this new item discrimination index could be recommended to researchers when investigating item discrimination in two-dimensional structures.

Recently, adaptive test approaches have become a viable alternative to traditional fixed-item tests. The main advantage of adaptive tests is that they reach desired measurement precision with fewer items. However, fewer items mean that each item has a more significant effect on ability estimation and therefore those tests are open to more consequential results from any flaw in an item. So, any items indicating differential item functioning (DIF) may play an important role in examinees' test scores. This study, therefore, aimed to investigate the effect of DIF items on the performance of computer adaptive and multi-stage tests. For this purpose, different test designs were tested under different test lengths and ratios of DIF items using Monte Carlo simulation. As a result, it was seen that computer adaptive test (CAT) designs had the best measurement precision over all conditions. When multi-stage test (MST) panel designs were compared, it was found that the 1-3-3 design had higher measurement precision in most of the conditions; however, the findings were not enough to say that 1-3-3 design performed better than the 1-2-4 design. Furthermore, CAT was found to be the least affected design by the increase of ratio of DIF items. MST designs were affected by that increment especially in the 10-item length test.

Simulation modeling is an effective tool for solving problems that cannot be explained analytically or when data cannot be collected. This is done by simulating the observed behavior of a problem under study using a computer program. In math education, this can develop knowledge and fundamental competencies of simulation modeling at a higher level and foster its applications in everyday life. This study created learning activities for secondary school students to simulate the probabilistic behavior of random events using Python. 28 grade 12 students took part in these activities using appropriate scaffolding strategies and a powerful mathematical tool, Python. After completing the activities, student competency in simulating the probabilistic behavior of random events with Python was evaluated using rubrics and the factors of student enjoyment, perceived value, interest, and self-efficacy were determined through a Likert-scale questionnaire. The assessment results showed that the activities had a positive effect on student competencies and emotions. The outcomes of the study can serve guidelines for teachers who are interested in expanding the results for further student development.

Item analysis is performed by developers as an integral part of the scale development process. Thus, items are excluded from the scale depending on the item analysis prior to the factor analysis. Existing item discrimination indices are calculated based on correlation, yet items with different response patterns are likely to have a similar item discrimination index. This study proposed a new item discrimination index that can be used in Likert type of scales and examined its effect on factor analysis results. For this purpose, simulative datasets were generated, and items were excluded from the analysis according to the 0.20, 0.30 and 0.35 item discrimination index criteria, and exploratory factor analysis was performed for a single factor. Accordingly, it was found that more variance could be explained by a single factor with fewer items compared to other discrimination indices when the 0.20 criterion of the slope coefficient was used as suggested in this study. Similar findings were obtained using the 0.35 criterion with other discrimination indices. In this context, it is recommended to use the slope coefficient as an additional discrimination index calculation method in the scale development process.

Financial professionals involved in divorce proceedings, whether for a client or an attorney, often use software to project the ability of a dependent spouse to earn income off of her separate estate. These projections have historically relied on static inputs and use a Monte Carlo simulation to illustrate the paths a portfolio might take. Within this study, the effects on dynamic income and expense changes on outcomes were examined. A comparison was made between the traditional Monte Carlo methods and Markov Chain Monte Carlo (MCMC) methods. Results using MCMC methods more closely approximated investment return distribution, and illustrated investable assets were the primary driver of long-term success, and not items such as spousal or child support. Practical implications for financial professionals, family law attorneys, judges, and clients are discussed as well as opportunities for future research.

Computerized Adaptive Testing (CAT) is a beneficial test technique that decreases the number of items that need to be administered by taking items in accordance with individuals' own ability levels. After the CAT applications were constructed based on the unidimensional Item Response Theory (IRT), Multidimensional CAT (MCAT) applications have gained momentum with the improvement of multidimensional IRT (MIRT) models in recent years. Researchers often benefit from simulation studies in order to design the final adaptive testing application and to test the effectiveness of adaptive testing applications they developed with different methods. Recently, R has become one of the most widely used programming languages in Monte Carlo Simulation studies since it is a free and open-source software. The aims of this study are to present the MCAT simulation process step by step in the R environment, to examine the effects of the conditions that researchers can handle during the simulation process according to two different dimensional models, and to examine the effect of treating multidimensional structures as unidimensional structures on simulation results. In this direction, datasets generated in accordance with within-item dimensionality and between-item dimensionality models, MCAT simulation studies were constructed with different customizations, and MCAT simulation results were compared with unidimensional CAT simulation results. All commands required for each simulation example were explained and results were shared for each condition.

When working with multimodal Bayesian posterior distributions, Markov chain Monte Carlo (MCMC) algorithms have difficulty moving between modes, and default variational or mode-based approximate inferences will understate posterior uncertainty. And, even if the most important modes can be found, it is difficult to evaluate their relative weights in the posterior. Here we propose an approach using parallel runs of MCMC, variational, or mode-based inference to hit as many modes or separated regions as possible and then combine these using Bayesian stacking, a scalable method for constructing a weighted average of distributions. The result from stacking efficiently samples from multimodal posterior distribution, minimizes cross validation prediction error, and represents the posterior uncertainty better than variational inference, but it is not necessarily equivalent, even asymptotically, to fully Bayesian inference. We present theoretical consistency with an example where the stacked inference approximates the true data generating process from the misspecified model and a non-mixing sampler, from which the predictive performance is better than full Bayesian inference, hence the multimodality can be considered a blessing rather than a curse under model misspecification. We demonstrate practical implementation in several model families: latent Dirichlet allocation, Gaussian process regression, hierarchical regression, horseshoe variable selection, and neural networks.

Cognitive diagnosis models (CDMs) are the statistical framework for cognitive diagnostic assessment in education and psychology. They generally assume that subjects' latent attributes are dichotomous--mastery or nonmastery, which seems quite deterministic. As an alternative to dichotomous attribute mastery, attention is drawn to the use of a continuous attribute mastery format in recent literature. To obtain subjects' finer-grained attribute mastery for more precise diagnosis and guidance, an equivalent but more explicit form of the partial-mastery-deterministic inputs, noisy "and" gate (DINA) model (termed continuous attribute profile [CAP]-DINA form) is proposed in this article. Its parameters estimation algorithm based on this form using Bayesian techniques with Markov chain Monte Carlo algorithm is also presented. Two simulation studies are conducted then to explore its parameter recovery and model misspecification, and the results demonstrate that the CAP-DINA form performs robustly with satisfactory efficiency in these two aspects. A real data study of the English test also indicates it has a better model fit than DINA.

Hybrid autoregressive-latent growth structural equation models for longitudinal data represent a synthesis of the autoregressive and latent growth modeling frameworks. Although these models are conceptually powerful, in practice they may struggle to separate autoregressive and growth-related processes during estimation. This confounding of change processes may, in turn, increase the risk of the models producing deceptively compelling results (i.e., models that fit excellently by conventional standards despite highly biased parameter estimates). Including additional time points provides models with more raw information about change, which could help improve process separability and the accuracy of parameter estimates to a degree. This study thus used Monte Carlo simulation methods to examine associations between change process separability, the number of time points in a model, and the consequences of misspecification, across three prominent hybrid autoregressive-latent growth models: the Latent Change Score model (LCS), the Autoregressive Latent Trajectory Model (ALT), and the Latent Growth Model with Structured Residuals (LGM-SR). Results showed that including more time points increased process separability and robustness to misspecification in the LCS and ALT, but typically not at a rate that would be practically feasible for most developmental researchers. Alternatively, regardless of how many time points were in the model process separability was high in the LGM-SR, as was robustness to misspecification. Overall, results suggest that the LGM-SR is the most effective of the three hybrid autoregressive-latent growth models considered here.

This study seeks to compare fixed and mixed effects models for the purposes of predictive classification in the presence of multilevel data. The first part of the study utilizes a Monte Carlo simulation to compare fixed and mixed effects logistic regression and random forests. An applied examination of the prediction of student retention in the public-use U.S. PISA data set was considered to verify the simulation findings. Results of this study indicate fixed effects models performed comparably with mixed effects models across both the simulation and PISA examinations. Results broadly suggest that researchers should be cognizant of the type of predictors and data structure being used, as these factors carried more weight than did the model type.

The present paper introduces a general multidimensional model to measure individual differences in learning within a single administration of a test. Learning is assumed to result from practicing the operations involved in solving the items. The model accounts for the possibility that the ability to learn may manifest differently for correct and incorrect responses, which allows for distinguishing different types of learning effects in the data. Model estimation and evaluation is based on a Bayesian framework. A simulation study is presented that examines the performance of the estimation and evaluation methods. The results show accuracy in parameter recovery as well as good performance in model evaluation and selection. An empirical study illustrates the applicability of the model to data from a logical ability test.

Back random responding (BRR) behavior is one of the commonly observed careless response behaviors. Accurately detecting BRR behavior can improve test validities. Yu and Cheng (2019) showed that the change point analysis (CPA) procedure based on weighted residual (CPA-WR) performed well in detecting BRR. Compared with the CPA procedure, the exponentially weighted moving average (EWMA) obtains more detailed information. This study equipped the weighted residual statistic with EWMA, and proposed the EWMA-WR method to detect BRR. To make the critical values adaptive to the ability levels, this study proposed the Monte Carlo simulation with ability stratification (MC-stratification) method for calculating critical values. Compared to the original Monte Carlo simulation (MC) method, the newly proposed MC-stratification method generated a larger number of satisfactory results. The performances of CPA-WR and EWMA-WR were evaluated under different conditions that varied in the test lengths, abnormal proportions, critical values and smoothing constants used in the EWMA-WR method. The results showed that EWMA-WR was more powerful than CPA-WR in detecting BRR. Moreover, an empirical study was conducted to illustrate the utility of EWMA-WR for detecting BRR.

The purpose of this study was to examine the effects of different data conditions on item parameter recovery and classification accuracy of three dichotomous mixture item response theory (IRT) models: the Mix1PL, Mix2PL, and Mix3PL. Manipulated factors in the simulation included the sample size (11 different sample sizes from 100 to 5000), test length (10, 30, and 50), number of classes (2 and 3), the degree of latent class separation (normal/no separation, small, medium, and large), and class sizes (equal vs. nonequal). Effects were assessed using root mean square error (RMSE) and classification accuracy percentage computed between true parameters and estimated parameters. The results of this simulation study showed that more precise estimates of item parameters were obtained with larger sample sizes and longer test lengths. Recovery of item parameters decreased as the number of classes increased with the decrease in sample size. Recovery of classification accuracy for the conditions with two-class solutions was also better than that of three-class solutions. Results of both item parameter estimates and classification accuracy differed by model type. More complex models and models with larger class separations produced less accurate results. The effect of the mixture proportions also differentially affected RMSE and classification accuracy results. Groups of equal size produced more precise item parameter estimates, but the reverse was the case for classification accuracy results. Results suggested that dichotomous mixture IRT models required more than 2,000 examinees to be able to obtain stable results as even shorter tests required such large sample sizes for more precise estimates. This number increased as the number of latent classes, the degree of separation, and model complexity increased.

A Monte Carlo simulation was performed to compare methods for handling missing data in growth mixture models. The methods considered in the current study were (a) a fully Bayesian approach using a Gibbs sampler, (b) full information maximum likelihood using the expectation-maximization algorithm, (c) multiple imputation, (d) a two-stage multiple imputation method, and (e) listwise deletion. Of the five methods, it was found that the Bayesian approach and two-stage multiple imputation methods generally produce less biased parameter estimates compared to maximum likelihood or single imputation methods, although key differences were observed. Similarities and disparities among methods are highlighted and general recommendations articulated.

There is a great deal of research about item response theory (IRT) conducted by simulations. Item and ability parameters are estimated with varying numbers of replications under different test conditions. However, it is not clear what the appropriate number of replications should be. The aim of the current study is to develop guidelines for the adequate number of replications in conducting Monte Carlo simulation studies involving unidimensional IRT models. For this aim, 192 simulation conditions which included four sample sizes, two test lengths, eight replication numbers, and unidimensional IRT models were generated. Accuracy and precision of item and ability parameter estimations and model fit values were evaluated by considering the number of replications. In this context, for the item and ability parameters; mean error, root mean square error, standard error of estimates, and for model fit; M[subscript 2], RMSEA[subscript 2], and Type I error rates were considered. The number of replications did not seem to influence the model fit, it was decisive in Type I error inflation and error prediction accuracy for all IRT models. It was concluded that to get more accurate results, the number of replications should be at least 625 in terms of accuracy of the Type I error rate estimation for all IRT models. Also, 156 replications and above can be recommended. Item parameter biases were examined, and the largest bias values were obtained from the 3PL model. It can be concluded that the increase in the number of parameters estimated by the model resulted in more biased estimates.

Monte Carlo simulation is a useful tool for researchers to estimated accuracy of a statistical model. It is usually used for investigating parameter estimation procedure or violation of assumption for some given conditions. To run a simulation either the paid software or open source but free program such as R is need to be used. For that, researchers must have a good knowledge about the theoretical procedures. This paper introduces the R package called MonteCarloSEM. The package helps to simulate and analyze data sets for some simulation condition such as sample size and normality for a given model. Also, an example is given to show how the functions within the package works.

In exploratory factor analysis, although the researchers decide which items belong to which factors by considering statistical results, the decisions taken sometimes can be subjective in case of having items with similar factor loadings and complex factor structures. The aim of this study was to examine the validity of classifying items into dimensions with exploratory graph analysis (EGA), which has been used in determining the number of dimensions in recent years and machine learning methods. A Monte Carlo simulation was performed with a total number of 96 simulation conditions including average factor loadings, sample size, number of items per dimension, number of dimensions, and distribution of data. Percent correct and Kappa concordance values were used in the evaluation of the methods. When the findings obtained for different conditions were evaluated together, it was seen that the machine learning methods gave results comparable to those of EGA. Machine learning methods showed high performance in terms of percent correct values, especially in small and medium-sized samples. In all conditions where the average factor loading was .70, BayesNet, Naive Bayes, RandomForest, and RseslibKnn methods showed accurate classification performances above 80% like EGA method. BayesNet, Simple Logistic and RBFNetwork methods also demonstrated acceptable or high performance under many conditions. In general, Kappa concordance values also supported these results. The results revealed that machine learning methods can be used for similar conditions to examine whether the distribution of items across factors is done accurately or not.

This study investigated the power of two-level hierarchical linear modeling (HLM) to explain variability in intervention effectiveness between participants in context of single-case experimental design (SCED) research. HLM is a flexible technique that allows the inclusion of participant characteristics (e.g., age, gender, and disability types) as moderators, and as such supplements visual analysis findings. First, this study empirically investigated the power to estimate intervention and moderator effects using Monte Carlo simulation techniques. The results indicate that larger values for the true effects and the number of participants resulted in a higher power. The more moderators added to the model, the more participants needed to detect the effects with sufficient power (i.e., power [greater than or equal to]0.80). When a model includes three moderators, at least 20 participants are required to capture the intervention effect and moderator effects with sufficient power. For that same condition, but only including one moderator, seven participants are sufficient. Specific recommendations for designing a SCED study with sufficient power to estimate intervention and moderator effects were provided. Second, this study introduced a newly developed user-friendly point and click Shiny tool, "PowerSCED." This tool assists applied SCED researchers in designing a SCED study that has sufficient power to detect intervention and moderator effects. To end, the use of HLM with the inclusion of moderators was demonstrated using two previously published SCED studies in the journal "School Psychology Quarterly."

Many statistical methods (estimators) are available for estimating the consensus value (or average effect) and heterogeneity variance in interlaboratory studies or meta-analyses. These estimators are all valid because they are developed from or supported by certain statistical principles. However, no estimator can be perfect and must have error or uncertainty (known as estimator uncertainty). For a given dataset, the consensus value and heterogeneity variance given by different estimators can often differ significantly. Consequently, the choice of different estimators can affect the conclusion of an interlaboratory study or meta-analysis. However, there is no universally accepted metric for determining which estimator is optimal among a set of candidate estimators. Instead of selecting and using a single estimator, this paper proposes an estimator-averaging approach to combine a set of individual estimators. The final averaged estimator is a linear combination of individual estimators, which accounts for three sources of uncertainties including the estimator uncertainty. Monte Carlo simulations were performed to examine the long-run performance of four individual estimators and the proposed averaged estimators. A case study: the determination of the Newtonian constant of gravitation is presented, where 10 individual estimators (eight frequentist weighted average methods and two Bayesian methods) are combined using the proposed estimator-averaging approach.

The meta-analytic-predictive (MAP) approach is a Bayesian method to incorporate historical controls in new trials that aims to increase the statistical power and reduce the required sample size. Here we investigate how to calculate the sample size of the new trial when historical data is available, and the MAP approach is used in the analysis. In previous applications of the MAP approach, the prior effective sample size (ESS) acted as a metric to quantify the number of subjects the historical information is worth. However, the validity of using the prior ESS in sample size calculation (i.e., reducing the number of randomized controls by the derived prior ESS) is questionable, because different approaches may yield different values for prior ESS. In this work, we propose a straightforward Monte Carlo approach to calculate the sample size that achieves the desired power in the new trial given available historical controls. To make full use of the available historical information to simulate the new trial data, the control parameters are not taken as a point estimate but sampled from the MAP prior. These sampled control parameters and the MAP prior based on the historical data are then used to derive the statistical power for the treatment effect and the resulting required sample size. The proposed sample size calculation approach is illustrated with real-life data sets with different outcomes from three studies. The results show that this approach to calculating the required sample size for the MAP analysis is straightforward and generic.

A sample suffers range restriction (RR) when its variance is reduced comparing with its population variance and, in turn, it fails representing such population. If the RR occurs over the latent factor, not directly over the observed variable, the researcher deals with an indirect RR, common when using convenience samples. This work explores how this problem affects different outputs of the factor analysis: multivariate normality (MVN), estimation process, goodness-of-fit, recovery of factor loadings, and reliability. In doing so, a Monte Carlo study was conducted. Data were generated following the linear selective sampling model, simulating tests varying their sample size (N = 200 and 500 cases), test size (J = 6, 12, 18, and 24 items), loading size (L = 0.50, 0.70, and 0.90), and restriction size (from R = 1, 0.90, 0.80, and so on till 0.10 selection ratio). Our results systematically suggest that an interaction between decreasing the loading size and increasing the restriction size affects the MVN assessment, obstructs the estimation process, and leads to an underestimation of the factor loadings and reliability. However, most of the MVN tests and most of the fit indices employed were nonsensitive to the RR problem. We provide some recommendations to applied researchers.

The purpose of this study was to evaluate the degree of classification quality in the basic latent class model when covariates are either included or are not included in the model. To accomplish this task, Monte Carlo simulations were conducted in which the results of models with and without a covariate were compared. Based on these simulations, it was determined that models without a covariate better predicted the number of classes. These findings in general supported the use of the popular three-step approach; with its quality of classification determined to be more than 70% under various conditions of covariate effect, sample size, and quality of indicators. In light of these findings, the practical utility of evaluating classification quality is discussed relative to issues that applied researchers need to carefully consider when applying latent class models.

Oral reading fluency (ORF), used by teachers and school districts across the country to screen and progress monitor at-risk readers, has been documented as a good indicator of reading comprehension and overall reading competence. In traditional ORF administration, students are given one minute to read a grade-level passage, after which the assessor calculates the words correct per minute (WCPM) fluency score by subtracting the number of incorrectly read words from the total number of words read aloud. As part of a larger effort to develop an improved ORF assessment system, this study expands on and demonstrates the performance of a new model-based estimate of WCPM based on a recently developed latent-variable psychometric model of speed and accuracy for ORF data. The proposed method was applied to a data set collected from 58 fourth-grade students who read four passages (a total of 260 words). The proposed model-based WCPM scores were also evaluated through a simulation study with respect to sample size and number of passages read. [For the corresponding grantee submission, see ED603990.]

Response time models (RTMs) are of increasing interest in educational and psychological testing. This article focuses on the lognormal model for response times, which is one of the most popular RTMs. Several existing statistics for testing normality and the fit of factor analysis models are repurposed for testing the fit of the lognormal model. A simulation study and two real data examples demonstrate the usefulness of the statistics. The Shapiro-Wilk test of normality and a "z"-test for factor analysis models were the most powerful in assessing the misfit of the lognormal model. [For the corresponding grantee submission, see ED603049.]

Fixed effects (FE) panel models have been used extensively in the past, as those models control for all stable heterogeneity between units. Still, the conventional FE estimator relies on the assumption of parallel trends between treated and untreated groups. It returns biased results in the presence of heterogeneous slopes or growth curves that are related to the parameter of interest (e.g., selection into treatment is based on individual growth of the outcome). In this study, we derive the bias in conventional FE models and show that fixed effects individual slope (FEIS) models can overcome this problem. This is a more general version of the conventional FE model, which accounts for heterogeneous slopes or trends, thereby providing a powerful tool for panel data and other multilevel data in general. We propose two versions of the Hausman test that can be used to identify misspecification in FE models. The performance of the FEIS estimator and the specification tests is evaluated in a series of Monte Carlo experiments. Using the examples of the marital wage premium and returns to preschool education (Head Start), we demonstrate how taking heterogeneous effects into account can seriously change the conclusions drawn from conventional FE models. Thus, we propose to test for bias in FE models in practical applications and to apply FEIS if indicated by the specification tests.

Meta-analytic models for dependent effect sizes have grown increasingly sophisticated over the last few decades, which has created challenges for a priori power calculations. We introduce power approximations for tests of average effect sizes based upon several common approaches for handling dependent effect sizes. In a Monte Carlo simulation, we show that the new power formulas can accurately approximate the true power of meta-analytic models for dependent effect sizes. Lastly, we investigate the Type I error rate and power for several common models, finding that tests using robust variance estimation provide better Type I error calibration than tests with model-based variance estimation. We consider implications for practice with respect to selecting a working model and an inferential approach.