Search

The relative advantages and disadvantages of sum scores and estimated factor scores are issues of concern for substantive research in psychology. Recently, while championing estimated factor scores over sum scores, McNeish offered a trenchant rejoinder to an article by Widaman and Revelle, which had critiqued an earlier paper by McNeish and Wolf. In the recent contribution, McNeish misrepresented a number of claims by Widaman and Revelle, rendering moot his criticisms of Widaman and Revelle. Notably, McNeish chose to avoid confronting a key strength of sum scores stressed by Widaman and Revelle-the greater comparability of results across studies if sum scores are used. Instead, McNeish pivoted to present a host of simulation studies to identify relative strengths of estimated factor scores. Here, we review our prior claims and, in the process, deflect purported criticisms by McNeish. We discuss briefly issues related to simulated data and empirical data that provide evidence of strengths of each type of score. In doing so, we identified a second strength of sum scores: superior cross-validation of results across independent samples of empirical data, at least for samples of moderate size. We close with consideration of four general issues concerning sum scores and estimated factor scores that highlight the contrasts between positions offered by McNeish and by us, issues of importance when pursuing applied research in our field.

Measurement is fundamental to all research in psychology and should be accorded greater scrutiny than typically occurs. Among other claims, McNeish and Wolf (Thinking twice about sum scores. Behavior Research Methods, 52, 2287-2305) argued that use of sum scores (a) implies that a highly constrained latent variable model underlies items comprising a scale, and (b) may misrepresent or bias relations with other criteria. The central claim by McNeish and Wolf that use of sum scores requires the assumption that a parallel test model underlies item responses is incorrect and without psychometric merit. Instead, if a set of items is unidimensional, estimators of reliability are available even if the factor model underlying the set of items does not have a highly constrained form. Thus, dimensionality of a set of items is the key issue, and whether strict constraints on parameter estimates do or do not hold dictate the appropriate way to estimate reliability. McNeish and Wolf also claimed that more precise forms of scoring, such as estimating factor scores, would be preferable to sum scores. We provide analytic bases for reliability estimation and then provide several demonstrations of reliability estimation and the relative advantages of sum scores and factor scores. We contend that several claims by McNeish and Wolf are questionable and that, as a result, multiple recommendations they made and conclusions they drew are incorrect. The upshot is that, once the dimensional structure of a set of items is verified, sum scores often have a solid psychometric basis and therefore are frequently quite adequate for psychological research.

A genuine interest in science is an important part of scientific literacy, and thus a critical goal for science education. Recent studies, however, have found that school science has not been effective in meeting this goal, an important reason for which is the lack of knowledge about what makes science interesting (or not) to the students. Using instructional episodes as the unit of analysis, this study investigated the effects of learning environment elements (content topic, activity, and learning goal) on student interest in science. The findings indicated that when judging the interestingness of an instructional episode, students focused primarily on the form of activity rather than content topic and learning goal. Activities that were "hands-on" in nature and allowed for engagement with technology elicited higher interest. This study highlights the need to place more emphasis on the role of activity in constructing interesting learning environments, and in the mean time, suggests that student science interest could be improved by making changes to relatively easy-to-manipulate aspects of learning environments, such as those examined in the study. (Contains 3 figures and 7 tables.)

The structure of the Hypomanic Personality Scale was explored in a sample of young adults (N = 884); resulting structures were validated on subsamples with measures of personality traits, internalizing symptoms, and externalizing behaviors. Hierarchical cluster analysis and estimates of general factor saturation suggested the presence of a weak general factor, as well as 3 subordinate clusters. The 3-cluster subscales (Social Vitality, Mood Volatility, and Excitement) were moderately correlated but conceptually distinct and were correlated in opposing patterns with a range of personality and psychopathology measures. Results show that cluster subscales suppress total score associations with validity measures. Recommendations are made to analyze Hypomanic Personality Scale data by subscale, and theoretical implications are discussed. (Contains 1 figure, 4 tables and 4 footnotes.)

There are three fundamental problems in Sijtsma ("Psychometrika," 2008): (1) contrary to the name, the glb is not the greatest lower bound of reliability but rather is systematically less than omega[subscript t] (McDonald, "Test theory: A unified treatment," Erlbaum, Hillsdale, 1999), (2) we agree with Sijtsma that when considering how well a test measures one concept, alpha is not appropriate, but recommend omega[subscript t] rather than the glb, and (3) the end user needs procedures that are readily available in open source software.

One of the most serious difficulties in the estimation of omega[subscript hierarchical] ([omega][subscript h]) has not been previously studied. Although it is true that one can impose a set of identifying constraints on one's solution to derive an estimate of [omega][subscript h] in the two-group factor case, there is the potential for introducing bias into the process of estimating [omega][subscript h], depending on the extent to which one's set of identifying constraints corresponds to the structure that actually underlies the scale. The authors demonstrate that this potential bias is only considerable when the ratio of the covariance between items loading on different group factors to the covariance between items loading on the same group factor is low. Four methods for estimating [omega][subscript h] are compared to each other, and the results suggest that Cronbach's alpha and a method based on the first unrotated principal factor should be rejected as estimates of [omega][subscript h]. (Contains 7 tables, 2 figures and 1 note.)

The extent to which a scale score generalizes to a latent variable common to all of the scale's indicators is indexed by the scale's general factor saturation. Seven techniques for estimating this parameter--omega[hierarchical] (omega[subscript h])--are compared in a series of simulated data sets. Primary comparisons were based on 160 artificial data sets simulating perfectly simple and symmetric structures that contained four group factors, and an additional 200 artificial data sets confirmed large standard deviations for two methods in these simulations when a general factor was absent. Major findings were replicated in a series of 40 additional artificial data sets based on the structure of a real scale widely believed to contain three group factors of unequal size and less than perfectly simple structure. The results suggest that alpha and methods based on either the first unrotated principal factor or component should be rejected as estimates of omega[subscript h]. (Contains 5 tables and 2 notes.)

We make theoretical comparisons among five coefficients--Cronbach's [alpha], Revelle's [beta], McDonald's [omega][sub h], and two alternative conceptualizations of reliability. Though many end users and psychometricians alike may not distinguish among these five coefficients, we demonstrate formally their nonequivalence. Specifically, whereas there are conditions under which [alpha], [beta], and [omega][sub h] are equivalent to each other and to one of the two conceptualizations of reliability considered here, we show that equality with this conceptualization of reliability and between [alpha] and [omega][sub h] holds only under a highly restrictive set of conditions and that the conditions under which [beta] equals [omega][sub h] are only somewhat more general. The nonequivalence of [alpha], [beta], and [omega][sub h] suggests that important information about the psychometric properties of a scale may be missing when scale developers and users only report [alpha] as is almost always the case.