Search

The purpose of this study was to investigate and compare the performance of a stepwise variable selection algorithm to traditional exploratory factor analysis. The Monte Carlo study included six factors in the design; the number of common factors; the number of variables explained by the common factors; the magnitude of factor loadings; the number of variables not explained by the common factors; the type of anomaly evidenced by the poorly explained variables; and sample size. The performance of the methods was evaluated in terms of selection and pattern accuracy, and bias and root mean squared error of the structure coefficients. Results indicate that the stepwise algorithm was generally ineffective at excluding anomalous variables from the factor model. The poor selection accuracy of the stepwise approach suggests that it should be avoided.

A loading matrix has perfect simple structure if each row has at most one nonzero element. It is shown that if there is an orthogonal rotation of an initial loading matrix that has perfect simple structure, then orthomax rotation with 0 ≤γ ≤ 1 of the initial loading matrix will produce the perfect simple structure. In particular, varimax and quartimax will produce rotations with perfect simple structure whenever they exist.

Factor analysis is regularly used for analyzing survey data. Missing data, data with outliers and consequently nonnormal data are very common for data obtained through questionnaires. Based on covariance matrix estimates for such nonstandard samples, a unified approach for factor analysis is developed. By generalizing the approach of maximum likelihood under constraints, statistical properties of the estimates for factor loadings and error variances are obtained. A rescaled Bartlett-corrected statistic is proposed for evaluating the number of factors. Equivariance and invariance of parameter estimates and their standard errors for canonical, varimax, and normalized varimax rotations are discussed. Numerical results illustrate the sensitivity of classical methods and advantages of the proposed procedures.

A very general algorithm for orthogonal rotation is identified. It is shown that when an algorithm parameterα is sufficiently large the algorithm converges monotonically to a stationary point of the rotation criterion from any starting value. Because a sufficiently largeα is in general hard to find, a modification that does not require it is introduced. Without this requirement the modified algorithm is not only very general, but also very simple. Its implementation involves little more than computing the gradient of the rotation criterion. While the modified algorithm converges monotonically from any starting value, it is not guaranteed to converge to a stationary point. It, however, does so in all of our examples. While motivated by the rotation problem in factor analysis, the algorithms discussed may be used to optimize almost any function of a not necessarily square column-wise orthonormal matrix. A number of these more general applications are considered. Empirical examples show that the modified algorithm can be reasonably fast, but its purpose is to save an investigator's effort rather than that of his or her computer. This makes it more appropriate as a research tool than as an algorithm for established methods.

We investigate under what conditions the matrix of factor loadings from the factor analysis model with equal unique variances will give a good approximation to the matrix of factor loadings from the regular factor analysis model. We show that the two models will give similar matrices of factor loadings if Schneeweiss' condition, that the difference between the largest and the smallest value of unique variances is small relative to the sizes of the column sums of squared factor loadings, holds. Furthermore, we generalize our results and discus the conditions under which the matrix of factor loadings from the regular factor analysis model will be well approximated by the matrix of factor loadings from Joreskog's image factor analysis model. Especially, we discuss Guttman's condition (i.e., the number of variables increases without limit) for the two models to agree, in relation with the condition we have shown, and conclude that Schneeweiss' condition is a generalization of Guttman's condition. Some implications for practice are discussed.

Current practice in factor analysis typically involves analysis of correlation rather than covariance matrices. We study whether the standardz-statistic that evaluates whether a factor loading is statistically necessary is correctly applied in such situations and more generally when the variables being analyzed are arbitrarily rescaled. Effects of rescaling on estimated standard errors of factor loading estimates, and the consequent effect onz-statistics, are studied in three variants of the classical exploratory factor model under canonical, raw varimax, and normal varimax solutions. For models with analytical solutions we find that some of the standard errors as well as their estimates are scale equivariant, while others are invariant. For a model in which an analytical solution does not exist, we use an example to illustrate that neither the factor loading estimates nor the standard error estimates possess scale equivariance or invariance, implying that different conclusions could be obtained with different scalings. Together with the prior findings on parameter estimates, these results provide new guidance for a key statistical aspect of factor analysis.

Matrices of factor loadings are often rotated to simple structure. When more than one loading matrix is available for the same variables, the loading matrices can be compared after rotating them all (separately) to simple structure. An alternative procedure is to rotate them to optimal agreement, and then compare them. In the present paper techniques are described that combine these two procedures. Specifically, five techniques that combine the ideals of rotation to optimal agreement and rotation to simple structure are compared on the basis of contrived and empirical data. For the contrived data, it is assessed to what extent the rotations recover the underlying common structure. For both the contrived and the empirical data it is studied to what extent the techniques give well matching rotated matrices, to what extent these have a simple structure, and to what extent the most prominent parts of the different loading matrices agree. It was found that the simple procedure of combining a Generalized Procrustes Analysis (GPA) with Varimax on the mean of the matched loading matrices performs very well on all criteria, and, for most purposes, offers an attractive compromise of rotation to agreement and simple structure. In addition to this comparison, some technical improvements are proposed for Bloxom's rotation to simple structure and maximum similarity.

Factor analysis and principal components analysis (PCA) are often followed by an orthomax rotation to rotate a loading matrix to simple structure. The simple structure is usually defined in terms of the simplicity of the columns of the loading matrix. In Three-made PCA, rotational freedom of the so called core (a three-way array relating components for the three different modes) can be used similarly to find a simple structure of the core. Simple structure of the core can be defined with respect to ail three modes simultaneously, possibly with different emphases on the different modes. The present paper provides a fully flexible approach for orthomax rotation of the core to simple structure with respect to three modes simultaneously. Computationally, this approach relies on repeated (two-way) orthomax applied to supermatrices containing the frontal, lateral or horizontal slabs, respectively. The procedure is illustrated by means of a number of exemplary analyses. As a by-product, application of the Three-mode Orthomax procedures to two-way arrays is shown to reveal interesting relations with and interpretations of existing two-way simple structure rotation techniques.

Varimax rotation consists of iteratively rotating pairs of columns of a matrix to a maximal sum (over columns) of variances of squared elements of the matrix. Without loss of optimality, the two rotated columns can be permuted and/or reflected. Although permutations and reflections are harmless for each planar rotation per se, they can be harmful in Varimax rotation. Specifically, they often give rise to the phenomenon that certain pairs of columns are consistently skipped in the iterative process, whence Varimax will be terminated at a nonstationary point. The skipping phenomenon is demonstrated, and it is shown how to prevent it.

Procedures for oblique rotation of factors or principal components typically focus on rotating the pattern matrix such that it becomes optimally simple. An important oblique rotation method that does so is Harris and Kaiser's (1964) independent cluster (HKIC) rotation. In principal components analysis, a case can be made for interpreting the components on the basis of the component weights rather than on the basis of the pattern, so it seems desirable to rotate the components such that the weights rather than the pattern become optimally simple. In the present paper, it is shown that HKIC rotates the components such that both the pattern and the weights matrix become optimally simple. In addition, it is shown that the pattern resulting from HKIC rotation is columnwise proportional to the associated weights matrix, which implies that the interpretation of the components does not depend on whether it is based on the pattern or on the component weights matrix. It is also shown that the latter result only holds for HKIC rotation and slight modifications of it.

Factor analysis and principal component analysis are usually followed by simple structure rotations of the loadings. These rotations optimize a certain criterion (e.g., varimax, oblimin), designed to measure the degree of simple structure of the pattern matrix. Simple structure can be considered optimal if a (usually large) number of pattern elements is exactly zero. In the present paper, a class of oblique rotation procedures is proposed to rotate a pattern matrix such that it optimally resembles a matrix which has an exact simple pattern. It is demonstrated that this method can recover relatively complex simple structures where other well-known simple structure rotation techniques fail.

Several methods have been developed for the analysis of a mixture of qualitative and quantitative variables, and one, called PCAMIX, includes ordinary principal component analysis (PCA) and multiple correspondence analysis (MCA) as special cases. The present paper proposes several techniques for simple structure rotation of a PCAMIX solution based on the rotation of component scores and indicates how these can be viewed as generalizations of the simple structure methods for PCA. In addition, a recently developed technique for the analysis of mixtures of qualitative and quantitative variables, called INDOMIX, is shown to construct component scores (without rotational freedom) maximizing the quartimax criterion over all possible sets of component scores. A numerical example is used to illustrate the implication that when used for qualitative variables, INDOMIX provides axes that discriminate between the observation units better than do those generated from MCA.

The present paper contains a lemma which implies that varimax rotation can be interpreted as a special case of diagonalizing symmetric matrices as discussed in multidimensional scaling. It is shown that the solution by De Leeuw and Pruzansky is essentially equivalent to the solution by Kaiser. Necessary and sufficient conditions for maxima and minima are derived from first and second order partial derivatives. A counter-example by Gebhardt is reformulated and examined in terms of these conditions. It is concluded that Kaiser's method or, equivalently, the method by De Leeuw and Pruzansky is the most attractive method currently available for the problem at hand.

Kaiser's iterative algorithm for the varimax rotation fails when (a) there is a substantial cluster of test vectors near the middle of each bounding hyperplane, leading to non-bounding hyperplanes more heavily overdetermined than those at the boundaries of the configuration of test vectors, and/or (b) there are appreciably more thanm (m factors) tests whose loadings on one of the factors of the initialF-matrix, usually the first, are near-zero, leading to overdetermination of the hyperplane orthogonal to this initialF-axis before rotation. These difficulties are overcome by weighting the test vectors, giving maximum weights to those likely to be near the primary axes, intermediate weights to those likely to be near hyperplanes but not near primary axes, and near-zero weights to those almost collinear with or almost orthogonal to the first initialF-axis. Applications to the Promax rotation are discussed, and it is shown that these procedures solve Thurstone's hitherto intractable “invariant” box problem as well as other more common problems based on real data.

Measures of test parsimony and factor parsimony are defined. Minimizing their weighted sum produces a general rotation criterion for either oblique or orthogonal rotation. The quartimax, varimax and equamax criteria are special cases of the expression. Two new criteria are developed. One of these, the parsimax criterion, apparently gives excellent results. It is argued that one of the most important factors bearing on the choice of a rotation criterion for a particular problem is the amount of information available on the number of factors that should be rotated.

An objective method for the orthogonal rotation of factors which gives results closer to the graphic method is proposed. First, the fact that the varimax method does not always satisfy simple-structure criteria, e.g., the positive manifold and the level contributions of all factors, is pointed out. Next, the principles of our method which are based on “geometric vector” are discussed, and the computational procedures for this method are explained using Harman and Holzinger's eight physical variables. Finally, six numerical examples by our method are presented, and it is shown that they are very close to the factors obtained from empirical studies both in values and in signs.

The quartimax and varimax algorithms for orthogonal rotation attempt to maximize particular simplicity criteria by a sequence of two-factor rotations. Derivations of these algorithms have been fairly complex. A simple general theory for obtaining “two factor at a time” algorithms for any polynomial simplicity criteria satisfying a natural symmetry condition is presented. It is shown that the degree of any symmetric criterion must be a multiple of four. A basic fourth degree algorithm, which is applicable to all symmetric fourth degree criteria, is derived and applied using a variety of criteria. When used with the quartimax and varimax criteria the algorithm is mathematically identical to the standard algorithms for these criteria. A basic eighth degree algorithm is also obtained and applied using a variety of eighth degree criteria. In general the problem of writing a basic algorithm for all symmetric criteria of any specified degree reduces to the problem of maximizing a trigonometric polynomial of degree one-fourth that of the criteria.

Two related orthogonal analytic rotation criteria for factor analysis are proposed. Criterion I is based upon the principle that variables which appear on the same factor should be correlated. Criterion II is based upon the principle that variables which are uncorrelated should not appear on the same factor. The recommended procedure is to rotate first by criterion I, eliminate the minor factors, and then rerotate the remaining major factors by criterion II. An example is presented in which this procedure produced a rotational solution very close to expectations whereas a varimax solution exhibited certain distortions. A computer program is provided.

Solutions for simple structure obtained from empirical data through the use of the Oblimax, Quartimax, and Varimax procedures are matched with independently determined graphic solutions for the same data. The relative merits of the two analytic orthogonal solutions (Quartimax and Varimax) are discussed. Factors of employee morale are interpreted.

In this article the authors examine the effect of including alternate test forms in a factor matrix upon the validity of the resultant factor loadings, finding that in this particular instance the effect is negligible. Comparisons of the factor loadings derived from matrices in which only one of the alternate test forms is included with those in which both forms are included reveal practically no difference in the magnitude of either the original or rotated factor loadings, or in that of the computed communalities.

In general, the methods of factor analysis developed during the past five years are based on the reduction of the correlational matrix by successive steps. The first factor loadings are determined and eliminated from the correlational matrix, giving a residual matrix. This process is continued for successive factor loadings until the elements of the last obtained residual matrix may be regarded as due to chance. The method outlined in this paper assumes the maximum number of factorsm in the correlational matrix. Them factor vectors are solved for simultaneously. Once them factor vectors are found, any vectors having only negligible factor loadings may be discarded.

The varimax solution for Thurstone's classic Primary Mental Abilities study is presented. Comparisons between the factors of Thurstone's original subjectively rotated factor pattern, Zimmerman's subjectively revised solution, Wrigley, Saunders, and Neuhaus' quartimax results, and the present varimax factor matrix are made by finding correlations between factors defined by these four solutions. It is pointed out that any possible ultimate merit of the varimax solution should be based on its psychological meaningfulness and on the rationale of the varimax criterion—not on its relationship to the other studies.

An analytic criterion for rotation is defined. The scientific advantage of analytic criteria over subjective (graphical) rotational procedures is discussed. Carroll's criterion and the quartimax criterion are briefly reviewed; the varimax criterion is outlined in detail and contrasted both logically and numerically with the quartimax criterion. It is shown that thenormal varimax solution probably coincides closely to the application of the principle of simple structure. However, it is proposed that the ultimate criterion of a rotational procedure is factorial invariance, not simple structure—although the two notions appear to be highly related. The normal varimax criterion is shown to be a two-dimensional generalization of the classic Spearman case, i.e., it shows perfect factorial invariance for two pure clusters. An example is given of the invariance of a normal varimax solution for more than two factors. The oblique normal varimax criterion is stated. A computational outline for the orthogonal normal varimax is appended.

The present note contains a completing-the-squares type approach to the varimax rotation problem. This approach yields a direct proof of global optimality of a solution for optimal rotation in a plane. Because varimax rotation can be interpreted as diagonalization of a set of symmetric matrices, the present solution also applies to the diagonalization problem.

The author provides a full-fledged matrix derivation of Sherin's matrix formulation of Kaiser's varimax criterion. He uses matrix differential calculus in conjunction with the Hadamard (or Schur) matrix product. Two results on Hadamard products are presented.

Usually, an iterative procedure based on two-dimensional rotations is employed to find the varimax solution in factor analysis. A matrix is given where this procedure does not yield the maximum value of the varimax criterion. However, random orthogonal transformations of some matrices and subsequent varimax-rotation using the iterative procedure seem to indicate that usually no local maxima exist.

The derivative of Kaiser's Varimax criterion, if set to zero, yields a set of equations which are quite similar to those obtained for a least-squares problem of the “Procrustes” type. This similarity suggested an iterative technique for orthogonal rotation, dubbed “Varisim,” which was programmed for the IBM 7094 in FORTRAN. An empirical comparison between Varimax and Varisim, which was based on a number of data sets taken from the literature yielded three major results so far: (i) Varisim is slower than Varimax, roughly by a factor of 3, (ii) Varisim yields factors which in general contribute more evenly to the common test variance than Varimax factors, and which (iii) line up more closely with oblique configurations obtained with Binormamin than Varimax factors.

Kaiser has given the varimax criterion for the solution of the rotation problem in factor analysis as well as a practical computational procedure for maximizing this criterion. In the present paper, the maximization condition is shown as a matrix equation involving only the unknown orthogonal rotation matrix. This matrix equation can be solved iteratively as a sequence of symmetric eigenproblems.