Search

A number of psychometricians have suggested that parallel analysis (PA) tends to yield more accurate results in determining the number of factors in comparison with other statistical methods. Nevertheless, all too often PA can suggest an incorrect number of factors, particularly in statistically unfavorable conditions (e.g., small sample sizes and low factor loadings). Because of this, researchers have recommended using multiple methods to make judgments about the number of factors to extract. Implicit in this recommendation is that, when the number of factors is chosen based on PA, uncertainty nevertheless exists. We propose a Bayesian parallel analysis (B-PA) method to incorporate the uncertainty with decisions about the number of factors. B-PA yields a probability distribution for the various possible numbers of factors. We implement and compare B-PA with a frequentist approach, revised parallel analysis (R-PA), in the contexts of real and simulated data. Results show that B-PA provides relevant information regarding the uncertainty in determining the number of factors, particularly under conditions with small sample sizes, low factor loadings, and less distinguishable factors. Even if the indicated number of factors with the highest probability is incorrect, B-PA can show a sizable probability of retaining the correct number of factors. Interestingly, when the mode of the distribution of the probabilities associated with different numbers of factors was treated as the number of factors to retain, B-PA was somewhat more accurate than R-PA in a majority of the conditions.

This study examined accuracy on syllable-final (coda) consonants in newly-learned English-like nonwords to determine whether school-aged bilingual children may be more vulnerable to making errors on English-only codas than their monolingual, English-speaking peers, even at a stage in development when phonological accuracy in productions of familiar words is high. Bilingual Spanish-English-speaking second- graders (age 7-9) with typical development (n=40) were matched individually with monolingual peers on age, sex, and speech skills. Participants learned to name sea monsters as part of five computerized word learning tasks. Dependent t-tests revealed bilingual children were less accurate than monolingual children in producing codas unique to English; however, the groups demonstrated equivalent levels of accuracy on codas that occur in both Spanish and English. Results suggest that, even at high levels of English proficiency, bilingual Spanish-English-speaking children may demonstrate lower accuracy than their monolingual English-speaking peers on targets that pattern differently in their two languages. Differences between a bilingual’s two languages can be used to reveal targets that may be more vulnerable to error, which could be a result of cross-linguistic effects or more limited practice with English phonology.

The Home Observation for Measurement of the Environment (HOME) Inventory is designed to assess the quality and quantity of support, stimulation, and structure provided to children in the home environment. HOME has been widely used for research and applied purposes. We focused on an abbreviated version of the Early Adolescent HOME (EA-HOME-A) that was administered to 15-year-old adolescents and their parents ( N = 958) as part of the NICHD (National Institute of Child Health and Human Development) Study of Early Child Care and Youth Development. Our study had two objectives. First, we hypothesized and tested a bifactor model that specified a general factor in support of the use of the HOME total score and group factors for subsets of items in support of the content domain scores. Second, we applied structural equation modeling to relate the EA-HOME-A factors to outcome factors assessing maladaptive behaviors, autonomy, self-control, and cognitive–academic performance. The results supported the construct validity of the EA-HOME-A with respect to its internal structure as well as its correlates.

Past research suggests revised parallel analysis (R-PA) tends to yield relatively accurate results in determining the number of factors in exploratory factor analysis. R-PA can be interpreted as a series of hypothesis tests. At each step in the series, a null hypothesis is tested that an additional factor accounts for zero common variance among measures in the population. Integration of an effect size statistic—the proportion of common variance (PCV)—into this testing process should allow for a more nuanced interpretation of R-PA results. In this article, we initially assessed the psychometric qualities of three PCV statistics that can be used in conjunction with principal axis factor analysis: the standard PCV statistic and two modifications of it. Based on analyses of generated data, the modification that considered only positive eigenvalues ([Formula: see text]) overall yielded the best results. Next, we examined PCV using minimum rank factor analysis, a method that avoids the extraction of negative eigenvalues. PCV with minimum rank factor analysis generally did not perform as well as [Formula: see text], even with a relatively large sample size of 5,000. Finally, we investigated the use of [Formula: see text] in combination with R-PA and concluded that practitioners can gain additional information from [Formula: see text] and make more nuanced decision about the number of factors when R-PA fails to retain the correct number of factors.

Parallel analysis (PA) assesses the number of factors in exploratory factor analysis. Traditionally PA compares the eigenvalues for a sample correlation matrix with the eigenvalues for correlation matrices for 100 comparison datasets generated such that the variables are independent, but this approach uses the wrong reference distribution. The proper reference distribution of eigenvalues assesses the kth factor based on comparison datasets with k−1 underlying factors. Two methods that use the proper reference distribution are revised PA (R-PA) and the comparison data method (CDM). We compare the accuracies of these methods using Monte Carlo methods by manipulating the factor structure, factor loadings, factor correlations, and number of observations. In the 17 conditions in which CDM was more accurate than R-PA, both methods evidenced high accuracies (i.e.,>94.5%). In these conditions, CDM had slightly higher accuracies (mean difference of 1.6%). In contrast, in the remaining 25 conditions, R-PA evidenced higher accuracies (mean difference of 12.1%, and considerably higher for some conditions). We consider these findings in conjunction with previous research investigating PA methods and concluded that R-PA tends to offer somewhat stronger results. Nevertheless, further research is required. Given that both CDM and R-PA involve hypothesis testing, we argue that future research should explore effect size statistics to augment these methods.

Relationships between structural equation models are almost exclusively characterized as categorical. However, a simple yes–no answer as to whether 2 models are equivalent or nested (or other categorical descriptors of relationships) cannot address whether the models tend to fit similarly across a wide range of data, or how the characteristics of models and data affect the differences in fit. Therefore, such simple answers have limited value for model selection and evaluation. In this article we proposed a quantitative framework that allows for graphical display of results, incorporates the categorical distinctions in model similarity, is applicable to any model pairs, and offers a new method to generate data for studying model similarity. The framework is straightforward to implement and does not require complex analyses. The information this framework provides is diagnostic and can help researchers identify and explain how key characteristics in data and model parameters lead to (dis)similarity in fit.

In children with dyslexia, deficits in working memory have not been well-specified. We assessed second-grade children with dyslexia, with and without concomitant specific language impairment, and children with typical development. Immediate serial recall of lists of phonological (non-word), lexical (digit), spatial (location) and visual (shape) items were included. For the latter three modalities, we used not only standard span but also running span tasks, in which the list length was unpredictable to limit mnemonic strategies. Non-word repetition tests indicated a phonological memory deficit in children with dyslexia alone compared with those with typical development, but this difference vanished when these groups were matched for non-verbal intelligence and language. Theoretically important deficits in serial order memory in dyslexic children, however, persisted relative to matched typically developing children. The deficits were in recall of (1) spoken digits in both standard and running span tasks and (2) spatial locations, in running span only. Children with dyslexia with versus without language impairment, when matched on non-verbal intelligence, had comparable serial order memory, but differed in phonology. Because serial orderings of verbal and spatial elements occur in reading, the careful examination of order memory may allow a deeper understanding of dyslexia and its relation to language impairment. Copyright © 2017 John Wiley & Sons, Ltd.

This study investigated the structure of working memory in young school-age children by testing the fit of three competing theoretical models using a wide variety of tasks. The best fitting models were then used to assess the relationship between working memory and nonverbal measures of fluid reasoning (Gf) and visual processing (Gv) intelligence. One hundred sixty-eight English-speaking 7–9 year olds with typical development, from three states, participated. Results showed that Cowan’s three-factor embedded processes model fit the data slightly better than Baddeley and Hitch’s (1974) three-factor model (specified according to Baddeley, 1986) and decisively better than Baddeley’s (2000) four-factor model that included an episodic buffer. The focus of attention factor in Cowan’s model was a significant predictor of Gf and Gv. The results suggest that the focus of attention, rather than storage, drives the relationship between working memory, Gf, and Gv in young school-age children. Our results do not rule out the Baddeley and Hitch model, but they place constraints on both it and Cowan’s model. A common attentional component is needed for feature binding, running digit span, and visual short-term memory tasks; phonological storage is separate, as is a component of central executive processing involved in task manipulation. The results contribute to a zeitgeist in which working memory models are coming together on common ground (cf. Cowan, Saults, & Blume, 2014; Hu, Allen, Baddeley, & Hitch, 2016).

Purpose We compared novel word learning in 2nd-grade children with typical development who were Spanish–English bilinguals to English monolinguals to understand word learning in bilingual children. Method Children (monolinguals n = 167, bilinguals n = 76) engaged in 5 computer-based tasks that assessed word learning in 6 different contexts. The tasks measured children's ability to link novel names with novel objects/actions, make decisions about the accuracy of those names and objects/actions, recognize the semantic features of the objects/actions, and produce the novel names. For analysis, we used Bayesian repeated-measures analyses of covariance with Bayesian independent-samples t tests to clarify interactions. Results Monolingual and bilingual children differed in some, but not most, word learning situations. There was at least moderate evidence that bilingual children were less accurate at naming in 1 condition and at detecting mispronunciations in 3 of 6 contexts and were less accurate at judging semantic features of a referent when that referent was paired with orthographic information. Discussion Among children with typical development, there were few differences in novel word learning between monolingual and bilingual participants. When differences did occur, they suggested that bilinguals were more accepting of phonological variations of word productions than their monolingual peers.

Purpose Compared to children with typical development, children with dyslexia, developmental language disorder (DLD), or both often demonstrate working memory deficits. It is unclear how pervasive the deficits are or whether the deficits align with diagnostic category. The purpose of this study was to determine whether different working memory profiles would emerge on a comprehensive battery of central executive, phonological, visuospatial, and binding working memory tasks and whether these profiles were associated with group membership. Method Three hundred two 2nd graders with typical development, dyslexia, DLD, or dyslexia/DLD completed 13 tasks from the Comprehensive Assessment Battery for Children–Working Memory ( Gray, Alt, Hogan, Green, & Cowan, n.d. ) that assessed central executive, phonological, and visuospatial/attention components of working memory. Results Latent class analyses yielded 4 distinct latent classes: low overall (21%), average with high number updating (30%), average with low number updating (12%), and high overall (37%). Children from each disability group and children from the typically developing group were present in each class. Discussion Findings highlight the importance of knowing an individual child's working memory profile because working memory profiles are not synonymous with learning disabilities diagnosis. Thus, working memory assessments could contribute important information about children's cognitive function over and above typical psychoeducational measures.

The Persistent Academic Possible Selves Scale for Adolescents (PAPSS: 51items) was developed to measure diverse aspects of future-oriented motivation and future-oriented self-regulated learning in pursuing academic future goals. To validate the scale, (a) the internal structure of the PAPSS was examined (N = 495) and (b) the PAPSS was assessed in comparison with the Oyserman (2004) open-ended possible selves questionnaire as predictors of external measures (N = 302) using Multi-Goal Multi-Theory analysis. Overall, the results supported the construct validity of the PAPSS. The final factor structure included three persistent academic goal factors (improving grades, better student, and more attention in class) and five theory-based factors (social group identity, peer group identity, self-concept, motivational self-regulated learning, and performance). The structure demonstrated good model fit. Based on its relationships with external measures as well as theory, the PAPSS has advantages over other available measures.

Two models can be nonequivalent, but fit very similarly across a wide range of data sets. These near-equivalent models, like equivalent models, should be considered rival explanations for results of a study if they represent plausible explanations for the phenomenon of interest. Prior to conducting a study, researchers should evaluate plausible models that are alternatives to those hypothesized to evaluate whether they are near-equivalent or equivalent and, in so doing, address the adequacy of the study’s methodology. To assess the extent to which alternative models for a study are empirically distinguishable, we propose 5 indexes that quantify the degree of similarity in fit between 2 models across a specified universe of data sets. These indexes compare either the maximum likelihood fit function values or the residual covariance matrices of models. Illustrations are provided to support interpretations of these similarity indexes.

Purpose The purpose of this study was to examine differences in performance between monolingual and Spanish–English bilingual second graders (aged 7–9 years old) on executive function tasks assessing inhibition, shifting, and updating to contribute more evidence to the ongoing debate about a potential bilingual executive function advantage. Method One hundred sixty-seven monolingual English-speaking children and 80 Spanish–English bilingual children were administered 7 tasks on a touchscreen computer in the context of a pirate game. Bayesian statistics were used to determine if there were differences between the monolingual and bilingual groups. Additional analyses involving covariates of maternal level of education and nonverbal intelligence, and matching on these same variables, were also completed. Results Scaled-information Bayes factor scores more strongly favored the null hypothesis that there were no differences between the bilingual and monolingual groups on any of the executive function tasks. For 2 of the tasks, we found an advantage in favor of the monolingual group. Conclusions If there is a bilingual advantage in school-aged children, it is not robust across circumstances. We discuss potential factors that might counteract an actual advantage, including task reliability and environmental influences.

Parallel analysis (PA) is a useful empirical tool for assessing the number of factors in exploratory factor analysis. On conceptual and empirical grounds, we argue for a revision to PA that makes it more consistent with hypothesis testing. Using Monte Carlo methods, we evaluated the relative accuracy of the revised PA (R-PA) and traditional PA (T-PA) methods for factor analysis of tetrachoric correlations between items with binary responses. We manipulated five data generation factors: number of observations, type of factor model, factor loadings, correlation between factors, and distribution of thresholds. The R-PA method tended to be more accurate than T-PA, although not uniformly across conditions. R-PA tended to perform better relative to T-PA if the underlying model (a) was unidimensional but had some unique items, (b) had highly correlated factors, or (c) had a general factor as well as a group factor. In addition, R-PA tended to outperform T-PA if items had higher factor loadings and sample size was large. A major disadvantage of the T-PA method was that it frequently yielded inflated Type I error rates.

In the lead article, Davenport, Davison, Liou, & Love demonstrate the relationship among homogeneity, internal consistency, and coefficient alpha, and also distinguish among them. These distinctions are important because too often coefficient alpha—a reliability coefficient—is interpreted as an index of homogeneity or internal consistency. We argue that factor analysis should be conducted before calculating internal consistency estimates of reliability. If factor analysis indicates the assumptions underlying coefficient alpha are met, then it can be reported as a reliability coefficient. However, to the extent that items are multidimensional, alternative internal consistency reliability coefficients should be computed based on the parameter estimates of the factor model. Assuming a bifactor model evidenced good fit, and the measure was designed to assess a single construct, omega hierarchical—the proportion of variance of the total scores due to the general factor—should be presented. Omega—the proportion of variance of the total scores due to all factors—also should be reported in that it represents a more traditional view of reliability, although it is computed within a factor analytic framework. By presenting both these coefficients and potentially other omega coefficients, the reliability results are less likely to be misinterpreted.

Reliabilities of scores for experimental tasks are likely to differ from one study to another to the extent that the task stimuli change, the number of trials varies, the type of individuals taking the task changes, the administration conditions are altered, or the focal task variable differs. Given that reliabilities vary as a function of the design of these tasks and the characteristics of the individuals taking them, making inferences about the reliability of scores in an ongoing study based on reliability estimates from prior studies is precarious. Thus, it would be advantageous to estimate reliability based on data from the ongoing study. We argue that internal consistency estimates of reliability are underutilized for experimental task data and in many applications could provide this information using a single administration of a task. We discuss different methods for computing internal consistency estimates with a generalized coefficient alpha and the conditions under which these estimates are accurate. We illustrate use of these coefficients using data for three different tasks.

The objective was to offer guidelines for applied researchers on how to weigh the consequences of errors made in evaluating measurement invariance (MI) on the assessment of factor mean differences. We conducted a simulation study to supplement the MI literature by focusing on choosing among analysis models with different number of between-group constraints imposed on loadings and intercepts of indicators. Data were generated with varying proportions, patterns, and magnitudes of differences in loadings and intercepts as well as factor mean differences and sample size. Based on the findings, we concluded that researchers who conduct MI analyses should recognize that relaxing as well as imposing constraints can affect Type I error rate, power, and bias of estimates in factor mean differences. In addition, fit indexes can be misleading in making decisions about constraints of loadings and intercepts. We offer suggestions for making MI decisions under uncertainty when assessing factor mean differences.

Purpose Several novel techniques have been developed recently to assess the breadth of a speaker's vocabulary exhibited in a language sample. The specific aim of this study was to increase our understanding of the validity of the scores generated by different lexical diversity (LD) estimation techniques. Four techniques were explored: D , Maas, measure of textual lexical diversity, and moving-average type–token ratio. Method Four LD indices were estimated for language samples on 4 discourse tasks (procedures, eventcasts, story retell, and recounts) from 442 adults who are neurologically intact. The resulting data were analyzed using structural equation modeling. Results The scores for measure of textual lexical diversity and moving-average type–token ratio were stronger indicators of the LD of the language samples. The results for the other 2 techniques were consistent with the presence of method factors representing construct-irrelevant sources. Conclusion These findings offer a deeper understanding of the relative validity of the 4 estimation techniques and should assist clinicians and researchers in the selection of LD measures of language samples that minimize construct-irrelevant sources.

Sijtsma and van der Ark (2015) focused in their lead article on three frameworks for reliability estimation in nursing research: classical test theory (CTT), factor analysis (FA), and generalizability theory. We extend their presentation with particular attention to CTT and FA methods. We first consider the potential of yielding an overly negative or an overly positive assessment of reliability based on coefficient alpha. Next, we discuss other CTT methods for estimating reliability and how the choice of methods affects the interpretation of the reliability coefficient. Finally, we describe FA methods, which not only permit an understanding of a measure’s underlying structure but also yield a variety of reliability coefficients with different interpretations. On a more general note, we discourage reporting reliability as a two-choice outcome—unsatisfactory or satisfactory; rather, we recommend that nursing researchers make a conceptual and empirical argument about when a measure might be more or less reliable, depending on its use.

Different methods are reviewed for estimating reliability based on a single administration of a test consisting of ordered categorical items. The nonlinear SEM reliability coefficient appears more flexible than other methods because it is designed to assess the reliability of summed scores for ordered categorical items through the modeling of the item relationships. However, this method assumes that the variables underlying the ordered categorical items are normally distributed. We evaluated nonlinear and linear SEM reliability coefficients using a Monte Carlo study by manipulating factor structure, distribution of the continuous items, number of ordered categories, distribution of the ordered categorical items, and sample size. The nonlinear SEM reliability coefficient performed as well as or better than the linear SEM reliability coefficient in most conditions and was relatively robust to modest violations of the normality assumption. The nonlinear SEM reliability coefficient was substantially biased when categorical item distributions were extremely skewed and the skewness differed markedly across items. Bias in these coefficients may be diagnosable in practice by assessing model fit.

Purpose Orthographic facilitation describes the phenomenon in which a spoken word is produced more accurately when its corresponding written word is present during learning. We examined the orthographic facilitation effect in children with dyslexia because they have poor learning and recall of spoken words. We hypothesized that including orthography during spoken word learning would facilitate learning and recall. Method Children with dyslexia and children with typical development ( n = 46 per group), 7–9 years old, were matched for grade and nonverbal intelligence. Across 4 blocks of exposure in 1 session, children learned pairings between 4 spoken pseudowords and novel semantic referents in a modified paired-associate learning task. Two of the pairings were presented with orthography present, and 2 were presented with orthography absent. Recall of newly learned spoken words was assessed using a naming task. Results Both groups showed orthographic facilitation during learning and naming. During learning, both groups paired pseudowords and referents more accurately when orthography was present. During naming, children with typical development showed a large orthographic facilitation effect that increased across blocks. For children with dyslexia, this effect was present initially but then plateaued. Conclusions We demonstrate for the first time that children with dyslexia benefit from orthographic facilitation during spoken word learning. These findings have direct implications for teaching spoken vocabulary to children with dyslexia.

Bifactor models are commonly used to assess whether psychological and educational constructs underlie a set of measures. We consider empirical underidentification problems that are encountered when fitting particular types of bifactor models to certain types of data sets. The objective of the article was fourfold: (a) to allow readers to gain a better general understanding of issues surrounding empirical identification, (b) to offer insights into empirical underidentification with bifactor models, (c) to inform methodologists who explore bifactor models about empirical underidentification with these models, and (d) to propose strategies for structural equation model users to deal with underidentification problems that can emerge when applying bifactor models.

The Comprehensive Assessment Battery for Children - Working Memory (CABC-WM) is a computer-based battery designed to assess different components of working memory in young school-age children. Working memory deficits have been identified in children with language-based learning disabilities, including dyslexia1,2 and language impairment3,4, but it is not clear whether these children exhibit deficits in subcomponents of working memory, such as visuospatial or phonological working memory. The CABC-WM is administered on a desktop computer with a touchscreen interface and was specifically developed to be engaging and motivating for children. Although the long-term goal of the CABC-WM is to provide individualized working memory profiles in children, the present study focuses on the initial success and utility of the CABC-WM for measuring central executive, visuospatial, phonological loop, and binding constructs in children with typical development. Immediate next steps are to administer the CABC-WM to children with specific language impairment, dyslexia, and comorbid specific language impairment and dyslexia.

Traditional parallel analysis (T-PA) estimates the number of factors by sequentially comparing sample eigenvalues with eigenvalues for randomly generated data. Revised parallel analysis (R-PA) sequentially compares the kth eigenvalue for sample data to the kth eigenvalue for generated data sets, conditioned on k− 1 underlying factors. T-PA and R-PA are conceptualized as stepwise hypothesis-testing procedures and, thus, are alternatives to sequential likelihood ratio test (LRT) methods. We assessed the accuracy of T-PA, R-PA, and LRT methods using a Monte Carlo approach. Although no method was uniformly more accurate across all 180 conditions, the PA approaches outperformed LRT methods overall. Relative to T-PA, R-PA tended to perform better within the framework of hypothesis testing and to evidence greater accuracy in conditions with higher factor loadings.

The root mean square error of approximation (RMSEA) and the comparative fit index (CFI) are two widely applied indices to assess fit of structural equation models. Because these two indices are viewed positively by researchers, one might presume that their values would yield comparable qualitative assessments of model fit for any data set. When RMSEA and CFI offer different evaluations of model fit, we argue that researchers are likely to be confused and potentially make incorrect research conclusions. We derive the necessary as well as the sufficient conditions for inconsistent interpretations of these indices. We also study inconsistency in results for RMSEA and CFI at the sample level. Rather than indicating that the model is misspecified in a particular manner or that there are any flaws in the data, the two indices can disagree because (a) they evaluate, by design, the magnitude of the model's fit function value from different perspectives; (b) the cutoff values for these indices are arbitrary; and (c) the meaning of "good" fit and its relationship with fit indices are not well understood. In the context of inconsistent judgments of fit using RMSEA and CFI, we discuss the implications of using cutoff values to evaluate model fit in practice and to design SEM studies.

A standard strategy in structural equation modeling is to conduct multiple Lagrange multiplier (LM) tests after rejection of an initial model. Controlling for Type 1 error across these tests minimizes the likelihood of including unnecessary additional parameters in the model. Three methods for controlling Type I errors are evaluated using simulated data for factor analytic models: the standard approach which involves testing each parameter at the .05 level, a Bonferroni approach, and a simultaneous test procedure (STP). In the first part of the study, all samples were generated from a population in which all null hypotheses associated with the LM tests were correct. Three factors were manipu1,~ted: factor weights, sample size, and number of parameters in the specification search. The standard and the STP approaches yielded overly liberal and overly conservative familywise error rates, respectively, while the Bonferroni approach yielded error rates closer to the nominal level. In the second part of the study, data were generated in which one or more null hypotheses associated with the LM test were incorrect, and the number of parameters in the search was manipulated. Again the Bonferroni method was the best approach in controlling familywise: error rate, particularly when the alpha level was adjusted for the number of parameters evaluated at each step.

Numerous rules-of-thumb have been suggested for determining the minimum number of subjects required to conduct multiple regression analyses. These rules-of-thumb are evaluated by comparing their results against those based on power analyses for tests of hypotheses of multiple and partial correlations. The results did not support the use of rules-of-thumb that simply specify some constant (e.g., 100 subjects) as the minimum number of subjects or a minimum ratio of number of subjects (N) to number of predictors (m). Some support was obtained for a rule-of-thumb that N ≥ 50 + 8 m for the multiple correlation and N ≥104 + m for the partial correlation. However, the rule-of-thumb for the multiple correlation yields values too large for N when m ≥ 7, and both rules-of-thumb assume all studies have a medium-size relationship between criterion and predictors. Accordingly, a slightly more complex rule-of thumb is introduced that estimates minimum sample size as function of effect size as well as the number of predictors. It is argued that researchers should use methods to determine sample size that incorporate effect size.

Two contrasting views toward the evaluation of multiple tests of constraints and control of Type 1 errors in structural equation modeling are presented. (a) Exploring; data helps researchers make decisions about inclusion of relevant model parameters and control of Type 1 errors hinders this process. (b) Exploring data is not likely to yield meaningful models unless we can limit the process on the basis of methods and theory, and controlling Type I errors is a useful device: to force us to limit our searches. Also, in evaluating multiple tests of constraints for applications other than exploratory analyses, we should control for Type I errors as we do in testing multiple comparisons in analysis of variance. We argue for the second perspective and present examples to illustrate methods for controlling Type 1 errors when making model comparisons.

A number of psychometricians have argued for the use of parallel analysis to determine the number of factors. However, parallel analysis must be viewed at best as a heuristic approach rather than a mathematically rigorous one. The authors suggest a revision to parallel analysis that could improve its accuracy. A Monte Carlo study is conducted to compare revised and traditional parallel analysis approaches. Five dimensions are manipulated in the study: number of observations, number of factors, number of measured variables, size of the factor loadings, and degree of correlation between factors. Based on the results, the revised parallel analysis method, using principal axis factoring and the 95th percentile eigenvalue rule, offers promise.

Coefficient alpha is almost universally applied to assess reliability of scales in psychology. We argue that researchers should consider alternatives to coefficient alpha. Our preference is for structural equation modeling (SEM) estimates of reliability because they are informative and allow for an empirical evaluation of the assumptions underlying them. An example is presented to illustrate the advantages of SEM estimates of reliability.

Population and sample simulation approaches were used to compare the performance of parallel analysis using principal component analysis (PA-PCA) and parallel analysis using principal axis factoring (PA-PAF) to identify the number of underlying factors. Additionally, the accuracies of the mean eigenvalue and the 95th percentile eigenvalue criteria were examined. The 95th percentile criterion was preferable for assessing the first eigenvalue using either extraction method. In assessing subsequent eigenvalues, PA-PCA tended to perform as well as or better than PA-PAF for models with one factor or multiple minimally correlated factors; the relative performance of the mean eigenvalue and the 95th percentile eigenvalue criteria depended on the number of variables per factor. PA-PAF using the mean eigenvalue criterion generally performed best if factors were more than minimally correlated or if one or more strong general factors as well as group factors were present.

Previous studies suggest that results from specification searches, as typically employed in structural equation modeling, should not be used to reach strong research conclusions due to their poor reliability. Analyses of computer generated data indicate that search results can be sufficiently reliable for exploratory purposes with properly designed and analyzed studies.

The maximal split-half coefficient is computed by calculating all possible split-half reliability estimates for a scale and then choosing the maximal value as the reliability estimate. Osburn compared the maximal split-half coefficient with 10 other internal consistency estimates of reliability and concluded that it yielded the most consistently accurate estimate of reliability across the conditions explored. In this simulation study, the authors demonstrate that the maximal split-half coefficient can, under some conditions, badly overestimate reliability because of capitalization on chance and under other conditions appreciably underestimate reliability. Cross-validation can correct the former problem. A difficulty with the cross-validated coefficient is that it is a relatively inefficient estimator of reliability.

Reliability can be estimated using structural equation modeling (SEM). Two potential problems with this approach are that estimates may be unstable with small sample sizes and biased with misspecified models. A Monte Carlo study was conducted to investigate the quality of SEM estimates of reliability by themselves and relative to coefficient alpha. The SEM approach showed minimal bias when the model was correctly specified if items were relatively well defined by their underlying factor(s). They tended to demonstrate somewhat greater bias when the model was misspecified, particularly underspecified. Overall, SEM estimates were more stable than anticipated. Researchers are more likely to obtain accurate estimates of reliability using SEM by conducting large-sample studies with well-constructed scales and critically assessing model fit.

The general use of coefficient alpha to assess reliability should be discouraged on a number of grounds. The assumptions underlying coefficient alpha are unlikely to hold in practice, and violation of these assumptions can result in nontrivial negative or positive bias. Structural equation modeling was discussed as an informative process both to assess the assumptions underlying coefficient alpha and to estimate reliability

A method is presented for estimating reliability using structural equation modeling (SEM) that allows for nonlinearity between factors and item scores. Assuming the focus is on consistency of summed item scores, this method for estimating reliability is preferred to those based on linear SEM models and to the most commonly reported estimate of reliability, coefficient alpha.

This Monte Carlo study examined the impact of misspecifying the Σ matrix in longitudinal data analysis under both the multilevel model and mixed model frameworks. Under the multilevel model approach, under-specification and general-misspecification of the Σ matrix usually resulted in overestimation of the variances of the random effects (e.g., τ00, ττ11 ) and standard errors of the corresponding growth parameter estimates (e.g., SEβ 0, SEβ 1). Overestimates of the standard errors led to lower statistical power in tests of the growth parameters. An unstructured Σ matrix under the mixed model framework generally led to underestimates of standard errors of the growth parameter estimates. Underestimates of the standard errors led to inflation of the type I error rate in tests of the growth parameters. Implications of the compensatory relationship between the random effects of the growth parameters and the longitudinal error structure for model specification were discussed.

Multivariate methods for analyzing group differences in means on dependent variables include multivariate analysis of variance, discriminant analysis, and multivariate analysis of factor means. To make appropriate choices among these methods, researchers should understand the statistical models underlying them. We present these models using path diagrams within a structural equation modeling framework. Results for the different methods are presented for an example concerning coping with asthma.

We conducted the present study to investigate whether college students adjust their study strategies to meet the cognitive demands of testing, a metacognitive self-regulatory skill. Participants were randomly assigned to one of the two testing conditions. In one condition we told participants to study for a test that required deep-level cognitive processing and in the other to study for a test that required surface-level cognitive processing. Results suggested that college students adjust their study strategies so that they are in line with the cognitive processing demands of tests and that performance is mediated by the study strategies that are used.

It is well known that coefficient alpha is an estimate of reliability if its underlying assumptions are met and that it is a lower-bound estimate if the assumption of essential tau equivalency is violated. Very little literature addresses the assumption of uncorrelated errors among items and the effect of violating this assumption on alpha. True score models are proposed that can account for correlated errors. These models allow random measurement errors on earlier items to affect directly or indirectly scores on later items. Coefficient alpha may yield spuriously high estimates of reliability if these true score models reflect item responding. In practice, it is important to differentiate these models from models in which the errors are correlated because 1 or more factors have been left unspecified. If the latter model is an accurate representation of item responding, the assumption of essential tau equivalency is violated and alpha is a lower-bound estimate of reliability.

Analyses of variance (ANOVA) with the general linear model (GLM) in many standard statistical packages use an Overparameter ized model, a model unfamiliar to most behavioral science researchers. Estimates and significance tests with GLM procedures are calculated by computing generalized inverses and estimates of estimable functions. Using simple examples, the authors discuss the concepts that underlie the solutions for 1-way and 2-way ANOVAs with Overparameterized models and illustrate how these models allow one to evaluate the research hypotheses. The authors also extend the discussion of Overparameterized models to a more general modeling approach than GLM, the general linear mixed model. Many students and researchers in the behavioral -ciences routinely conduct analyses of variance

Two Lagrange multiplier (LM) methods may be used in specification searches for adding parameters to models: one based on univariate LM tests and respecification of the model (LM‐respecified method) and the other based on a partitioning of multivariate LM tests (LM‐incremental method). These methods may result in extraneous parameters being included in models due to either sampling error or the model being misspecified. A 2‐stage specification search may be used to reduce errors due to misspecification. In the 1st stage, parameters are added to models based on LM tests to maximize fit. Second, parameters added in the 1st stage are deleted if they are no longer necessary to maintain model fit. Illustrations are presented to demonstrate that errors due to misspecification occur with the LM‐respecified method and are even more likely with the LM‐incremental approach. These illustrations also show how the deletion stage can help eliminate some of these errors.

Junior and senior high school teachers in core subject areas (n = 102) were randomly assigned to read 1 of 3 scenarios describing classrooms composed of students of different ability levels: all low-ability, all high-ability, or both high- and low-ability students. After reading the assigned scenario, teachers indicated how they would assign test items over the dimensions of item type and cognitive complexity for the students described. Teachers also gave estimates of difficulty and pass rate for different types of items. Teachers who read scenarios about a high-achieving class reported that they would construct tests with significantly fewer recognition than essay items, whereas teachers who read scenarios about a low-achieving or a mixed-ability class reported that they would construct tests with more recognition items than essay items. Teachers who read scenarios describing low- and mixed-ability classrooms included significantly more memorization and application items than higher order items.

This study examined whether a curriculum-embedded assessment, the Practical Assessment Exploration System (PAES), could be used with students with disabilities to predict job-related outcomes that were obtained 3 to 5 years later. How well the PAES predicted these outcomes was compared with two traditional paper-and-pencil aptitude measures. The sample consisted of 103 students with disabilities who were evaluated using the PAES in the 7th and 8th grades. The results indicated that the PAES was most strongly related to the job-related outcome measure that assesses level of support required on the job but also tended to be related to the other 2 criteria, salary and hours worked In contrast, the traditional aptitude measures were almost uniformly very weakly related to the outcome measures. The results offer some promise for the use of embedded assessment as an approach to evaluate student potential as well as a meaningful method to assess student progress.

This article investigates the effect of the number of item response categories on chi‐square statistics for confirmatory factor analysis to assess whether a greater number of categories increases the likelihood of identifying spurious factors, as previous research had concluded. Four types of continuous single‐factor data were simulated for a 20‐item test: (a) uniform for all items, (b) symmetric unimodal for all items, (c) negatively skewed for all items, or (d) negatively skewed for 10 items and positively skewed for 10 items. For each of the 4 types of distributions, item responses were divided to yield item scores with 2,4, or 6 categories. The results indicated that the chi‐square statistic for evaluating a single‐factor model was most inflated (suggesting spurious factors) for 2‐category responses and became less inflated as the number of categories increased. However, the Satorra‐Bentler scaled chi‐square tended not to be inflated even for 2‐category responses, except if the continuous item data ha...