Search

Sparse and outlier-robust Principal Component Analysis (PCA) has been a very active field of research recently. Yet, most existing methods apply PCA to a single dataset whereas multi-source data-i.e. multiple related datasets requiring joint analysis-arise across many scientific areas. We introduce a novel PCA methodology that simultaneously (i) selects important features, (ii) allows for the detection of global sparse patterns across multiple data sources as well as local source-specific patterns, and (iii) is resistant to outliers. To this end, we develop a regularization problem with a penalty that accommodates global-local structured sparsity patterns, and where the ssMRCD estimator is used as plug-in to permit joint outlier-robust analysis across multiple data sources. We provide an efficient implementation of our proposal via the Alternating Direction Method of Multiplier and illustrate its practical advantages in simulation and in applications.

Principal component analysis and factor analysis are fundamental multivariate analysis methods. In this paper a unified framework to connect them is introduced. Under a general latent variable model, we present matrix optimization problems from the viewpoint of loss function minimization, and show that the two methods can be viewed as solutions to the optimization problems with specific loss functions. Specifically, principal component analysis can be derived from a broad class of loss functions including the L2 norm, while factor analysis corresponds to a modified L0 norm problem. Related problems are discussed, including algorithms, penalized maximum likelihood estimation under the latent variable model, and a principal component factor model. These results can lead to new tools of data analysis and research topics., Comment: 24 pages, 2 figures

Principal Component Analysis and biplots are so well-established and readily implemented that it is just too tempting to give for granted their internal workings. In this note I get back to basics in comparing how PCA and biplots are implemented in base-R and contributed R packages, leveraging an implementation-agnostic understanding of the computational structure of each technique. I do so with a view to illustrating discrepancies that users might find elusive, as these arise from seemingly innocuous computational choices made under the hood. The proposed evaluation grid elevates aspects that are usually disregarded, including relationships that should hold if the computational rationale underpinning each technique is followed correctly. Strikingly, what is expected from these equivalences rarely follows without caveats from the output of specific implementations alone., Comment: Preprint. 20 pages, 3 figure, 5 tables. Currently submitted to the R Journal

Varimax factor rotations, while popular among practitioners in psychology and statistics since being introduced by H. Kaiser, have historically been viewed with skepticism and suspicion by some theoreticians and mathematical statisticians. Now, work by K. Rohe and M. Zeng provides new, fundamental insight: varimax rotations provably perform statistical estimation in certain classes of latent variable models when paired with spectral-based matrix truncations for dimensionality reduction. We build on this newfound understanding of varimax rotations by developing further connections to network analysis and spectral methods rooted in entrywise matrix perturbation analysis. Concretely, this paper establishes the asymptotic multivariate normality of vectors in varimax-transformed Euclidean point clouds that represent low-dimensional node embeddings in certain latent space random graph models. We address related concepts including network sparsity, data denoising, and the role of matrix rank in latent variable parameterizations. Collectively, these findings, at the confluence of classical and contemporary multivariate analysis, reinforce methodology and inference procedures grounded in matrix factorization-based techniques. Numerical examples illustrate our findings and supplement our discussion., Comment: 21 pages with bibliography and supplement

The present study investigates to what degree the common variance of the factor score predictor with the original factor, i.e., the determinacy coefficient or the validity of the factor score predictor, depends on the mean-difference between groups. When mean-differences between groups in the factor score predictor are eliminated by means of covariance analysis, regression, or group specific norms, this may reduce the covariance of the factor score predictor with the common factor. It is shown that in a one-factor model with the same group mean-difference on all observed variables, the common factor cannot be distinguished from a common factor representing the group mean-difference. It is also shown that for common factor loadings equal or larger than .60, the elimination of a d = .50 mean-difference between two groups in the factor score predictor leads to only small decreases of the determinacy coefficient. A compensation-factor k is proposed allowing for the estimation of the number of additional observed variables necessary to recover the size of the determinacy coefficient before elimination of a group mean-difference. It turns out that for factor loadings equal or larger than .60 only a few additional items are needed in order to recover the initial determinacy coefficient after the elimination of moderate or large group mean-differences.

This paper presents a novel approach to functional principal component analysis (FPCA) in Bayes spaces in the setting where densities are the object of analysis, but only few individual samples from each density are observed. We use the observed data directly to account for all sources of uncertainty, instead of relying on prior estimation of the underlying densities in a two-step approach, which can be inaccurate if small or heterogeneous numbers of samples per density are available. To account for the constrained nature of densities, we base our approach on Bayes spaces, which extend the Aitchison geometry for compositional data to density functions. For modeling, we exploit the isometric isomorphism between the Bayes space and the $\mathbb{L}^2$ subspace $\mathbb{L}_0^2$ with integration-to-zero constraint through the centered log-ratio transformation. As only discrete draws from each density are observed, we treat the underlying functional densities as latent variables within a maximum likelihood framework and employ a Monte Carlo Expectation Maximization (MCEM) algorithm for model estimation. Resulting estimates are useful for exploratory analyses of density data, for dimension reduction in subsequent analyses, as well as for improved preprocessing of sparsely sampled density data compared to existing methods. The proposed method is applied to analyze the distribution of maximum daily temperatures in Berlin during the summer months for the last 70 years, as well as the distribution of rental prices in the districts of Munich.

Introduction: Oblique Target-rotation in the context of exploratory factor analysis is a relevant method for the investigation of the oblique independent clusters model. It was argued that minimizing single cross-loadings by means of target rotation may lead to large effects of sampling error on the target rotated factor solutions. Method: In order to minimize effects of sampling error on results of Target-rotation we propose to compute the mean cross-loadings for each block of salient loadings of the independent clusters model and to perform target rotation for the block-wise mean cross-loadings. The resulting transformation-matrix is than applied to the complete unrotated loading matrix in order to produce mean Target-rotated factors. Results: A simulation study based on correlated independent factor models revealed that mean oblique Target-rotation resulted in smaller negative bias of factor inter-correlations than conventional Target-rotation based on single loadings, especially when sample size was small and when the number of factors was large. An empirical example revealed that the similarity of Target-rotated factors computed for small subsamples with Target-rotated factors of the total sample was more pronounced for mean Target-rotation than for conventional Target-rotation. Discussion: Mean Target-rotation can be recommended in the context of oblique independent factor models, especially for small samples. An R-script and an SPSS-script for this form of Target-rotation are provided in the Appendix.

The relevance of determinacy coefficients as indicators for the validity of factor score predictors has regularly been emphasized. Previous simulation studies revealed biased determinacy coefficients for factor score predictors based on categorical variables. Therefore, and because there are different possibilities to compute determinacy coefficients, the present study compared bias of determinacy coefficients for the best linear factor score predictor and for a correlation-preserving factor score predictor based on confirmatory factor models with observed variables with 2, 4, 6, and 8 categories and maximum likelihood estimation, diagonally weighted least squares estimation, and Bayesian estimation. Positive bias was found when data were based on variables with two categories, population factors were correlated, and when there were unmodeled cross-loadings. Based on the results, the correction for sampling error, the use of maximum likelihood or Bayesian parameters, and data with at least four categories are recommended to avoid overestimation of parameter-based determinacy coefficients.

Regression factor score predictors have the maximum factor score determinacy, i.e., the maximum correlation with the corresponding factor, but they do not have the same inter-correlations as the factors. As it might be useful to compute factor score predictors that have the same inter-correlations as the factors, correlation-preserving factor score predictors have been proposed. However, correlation-preserving factor score predictors have smaller correlations with the corresponding factors (factor score determinacy) than regression factor score predictors. Thus, higher factor score determinacy goes along with bias of the inter-correlations and unbiased inter-correlations go along with lower factor score determinacy. The aim of the present study was therefore to investigate the size of the trade-off between factor score determinacy and bias of inter-correlations by means of a simulation study. It turns out that under several conditions very small gains of factor score determinacy of the regression factor score predictor go along with a large bias of inter-correlations. Instead of using the regression factor score predictor by default, it is proposed to check whether substantial bias of inter-correlations can be avoided without substantial loss of factor score determinacy by using a correlation-preserving factor score predictor. A syntax that allows to compute correlation-preserving factor score predictors from regression factor score predictors and to compare factor score determinacy and inter-correlations of the factor score predictors is given in the Appendix.

We analyze the prediction error of principal component regression (PCR) and prove high probability bounds for the corresponding squared risk conditional on the design. Our first main result shows that PCR performs comparably to the oracle method obtained by replacing empirical principal components by their population counterparts, provided that an effective rank condition holds. On the other hand, if the latter condition is violated, then empirical eigenvalues start to have a significant upward bias, resulting in a self-induced regularization of PCR. Our approach relies on the behavior of empirical eigenvalues, empirical eigenvectors and the excess risk of principal component analysis in high-dimensional regimes., Comment: 25 pages. arXiv admin note: text overlap with arXiv:1811.02998

Mean plausible values can be computed when Bayesian structural equation modeling (BSEM) is performed. As mean plausible values do not preserve the inter-factor correlations, they yield path coefficients that are different from the estimated path coefficients of the model. As it might be of interest to perform exactly the same prediction on the level of plausible values that has been estimated by BSEM, correlation-preserving mean plausible values were proposed. An example for the computation of the correlation preserving mean plausible values is given and the corresponding syntax is given in the Appendix.

The Bayesian information criterion (BIC), defined as the observed data log likelihood minus a penalty term based on the sample size $N$, is a popular model selection criterion for factor analysis with complete data. This definition has also been suggested for incomplete data. However, the penalty term based on the `complete' sample size $N$ is the same no matter whether in a complete or incomplete data case. For incomplete data, there are often only $N_i

This study aims to find out the current status and prospects of using a bicycle as a mode for commuting within Dhaka city. Bicycling is a very sustainable mode of transport but unfortunately is used very less by the commuters of Dhaka. There has been a lot of factors affecting the choice of bicycle to commute. This study was aimed to find out what factors could motivate the commuters of Dhaka to use a bicycle as a mode of transport. For determining the motivators, a survey was administered among the commuters of Dhaka city in which the respondents were asked to answer how certain factors would affect their choice to use a bicycle to commute. A Likert scale was used in the survey and the responses were analyzed, from which the top motivators were found. The Motivators were then grouped together using exploratory factor analysis to support possible policy making. Four factors were extracted using the method. The factors were named Additional Perks, General benefits, Personal Benefits, and Infrastructural benefits, Comment: 6 Pages, 2 tables, Conference

Effects of performing R-factor analysis of observed variables based on population models comprising R- and Q-factors were investigated. It was noted that estimating a model comprising R- and Q-factors has to face loading indeterminacy beyond rotational indeterminacy. Although R-factor analysis of data based on a population model comprising R- and Q-factors is nevertheless possible, this may lead to model error. Accordingly, even in the population, the resulting R-factor loadings are not necessarily close estimates of the original population R-factor loadings. It was shown in a simulation study that large Q-factor variance induces an increase of the variation of R-factor loading estimates beyond chance level. The results indicate that performing R-factor analysis with data based on a population model comprising R- and Q-factors may result in substantial loading bias. Tests of the multivariate kurtosis of observed variables are proposed as an indicator of possible Q-factor variance in observed variables as a prerequisite for R-factor analysis.

Principal component analysis (PCA) is a popular dimension reduction technique for vector data. Factored PCA (FPCA) is a probabilistic extension of PCA for matrix data, which can substantially reduce the number of parameters in PCA while yield satisfactory performance. However, FPCA is based on the Gaussian assumption and thereby susceptible to outliers. Although the multivariate $t$ distribution as a robust modeling tool for vector data has a very long history, its application to matrix data is very limited. The main reason is that the dimension of the vectorized matrix data is often very high and the higher the dimension, the lower the breakdown point that measures the robustness. To solve the robustness problem suffered by FPCA and make it applicable to matrix data, in this paper we propose a robust extension of FPCA (RFPCA), which is built upon a $t$-type distribution called matrix-variate $t$ distribution. Like the multivariate $t$ distribution, the matrix-variate $t$ distribution can adaptively down-weight outliers and yield robust estimates. We develop a fast EM-type algorithm for parameter estimation. Experiments on synthetic and real-world datasets reveal that RFPCA is compared favorably with several related methods and RFPCA is a simple but powerful tool for matrix-valued outlier detection., Comment: 37 pages

In this paper, we examine the materiality of ESG on country creditworthiness from a credit risk and fundamental analysis viewpoint. We first determine the ESG indicators that are most relevant when it comes to explaining the sovereign bond yield, after controlling the effects of traditional fundamental variables such as economic strength and credit rating. We also emphasize the major themes that are directly useful for investors when assessing the country risk premium. At the global level, we notice that these themes mainly belong to the E and G pillars. Those results confirm that extra-financial criteria are integrated into bond pricing. However, we also identify a clear difference between high-and middle-income countries. Indeed, whereas the S pillar is lagging for the highest income countries, it is nearly as important as the G pillar for the middle-income ones. Second, we determine which ESG metrics are indirectly valuable for assessing a country's solvency. More precisely, we attempt to infer credit rating solely from extra-financial criteria, that is the ESG indicators that are priced in by credit rating agencies. We find that there is no overlap between the set of indicators that predict credit ratings and those that directly explain sovereign bond yields. The results also highlight the importance of the G and S pillars when predicting credit ratings. The E pillar is lagging, suggesting that credit rating agencies are undermining the impact of climate change and environmental topics on country creditworthiness. This is consistent with the traditional view that social and governance issues are the main drivers of the sovereign risk, because they are more specific and less global than environmental issues. Finally, taking these different results together, this research shows that opposing extra-financial and fundamental analysis does not make a lot of sense., Comment: 102 pages, 22 figures, 69 tables. This version has been removed by arXiv administrators due to a copyright claim by a 3rd party

In the context of Bayesian factor analysis, it is possible to compute mean plausible values, which might be used as covariates or predictors or in order to provide individual scores for the Bayesian latent variables. Previous simulation studies ascertained the validity of the plausible values by the mean squared difference of the plausible values and the generating factor scores. However, the generating factor scores are unknown in empirical studies so that an indicator that is solely based on model parameters is needed in order to evaluate the validity of factor score estimates in empirical studies. The coefficient of determinacy is based on model parameters and can be computed whenever Bayesian factor analysis is performed in empirical settings. Therefore, the central aim of the present simulation study was to compare the coefficient of determinacy based on model parameters with the correlation of mean plausible values with the generating factors. It was found that the coefficient of determinacy yields an acceptable estimate for the validity of mean plausible values. As for small sample sizes and a small salient loading size the coefficient of determinacy overestimates the validity, it is recommended to report the coefficient of determinacy together with a bias-correction in order to estimate the validity of mean plausible values in empirical settings.

The paper is devoted to the consequences of blind random selection of items from different item populations that might be based on completely uncorrelated factors for item inter-correlations and corresponding factor loadings. Based on the model of essentially parallel measurements, we explore the consequences of presenting items from different populations across individuals and items from identical populations within each individual for the factor model and item inter-correlations in the total population of individuals. Moreover, we explore the consequences of presenting items from different as well as identical item populations across and within individuals. We show that correlations can be substantial in the total population of individuals even when -- in subpopulations of individuals -- items are drawn from populations with uncorrelated factors. In order to address this challenge for the validity of a scale, we propose a method that helps to detect whether item inter-correlations result from different item populations in different subpopulations of individuals and evaluate the method by means of a simulation study. Based on the analytical results and on results from a simulation study, we provide recommendations for the detection of subpopulations of individuals responding to items from different item populations.

The availability of multi-omics data has revolutionized the life sciences by creating avenues for integrated system-level approaches. Data integration links the information across datasets to better understand the underlying biological processes. However, high-dimensionality, correlations and heterogeneity pose statistical and computational challenges. We propose a general framework, probabilistic two-way partial least squares (PO2PLS), which addresses these challenges. PO2PLS models the relationship between two datasets using joint and data-specific latent variables. For maximum likelihood estimation of the parameters, we implement a fast EM algorithm and show that the estimator is asymptotically normally distributed. A global test for testing the relationship between two datasets is proposed, and its asymptotic distribution is derived. Notably, several existing omics integration methods are special cases of PO2PLS. Via extensive simulations, we show that PO2PLS performs better than alternatives in feature selection and prediction performance. In addition, the asymptotic distribution appears to hold when the sample size is sufficiently large. We illustrate PO2PLS with two examples from commonly used study designs: a large population cohort and a small case-control study. Besides recovering known relationships, PO2PLS also identified novel findings. The methods are implemented in our R-package PO2PLS. Supplementary materials for this article are available online., Comment: 36 pages, 4 figures, Submitted to Journal of the American Statistical Association

Multiple Indicators Multiple Causes (MIMIC) models are type of structural equation models, a theory-based approach to confirm the influence of a set of exogenous causal variables on the latent variable, and also the effect of the latent variable on observed indicator variables. In a common MIMIC model, multiple indicators reflect the underlying latent variables/factors, and the multiple causes (observed predictors) affect latent variables/factors. Basic assumptions of MIMIC are clearly violated in case of a variable being both an indicator and a cause, i.e. in the presence of reverse causality. Furthermore, the model is then unidentified. To resolve the situation, which can arise frequently, and as MIMIC estimation lacks closed form solutions for parameters we utilize a version of Bollen's (1996) 2SLS estimator for structural equation models combined with J\"oreskog (1970)'s method of the analysis of covariance structures to derive a new, 2SLS estimator for MIMIC models. Our 2SLS empirical estimation is based on static MIMIC specification but we point also to dynamic/error-correction MIMIC specification and 2SLS solution for it. We derive basic asymptotic theory for static 2SLS-MIMIC, present a simulation study and apply findings to an interesting empirical case of estimating precarious status of older workers (using dataset of Survey of Health, Ageing and Retirement in Europe) which solves an important issue of the definition of precarious work as a multidimensional concept, not modelled adequately so far., Comment: 21 pages, 3 figures

Score Predictor Factor Analysis (SPFA) was introduced as a method that allows to compute factor score predictors that are -- under some conditions -- more highly correlated with the common factors resulting from factor analysis than the factor score predictors computed from the common factor model. In the present study, we investigate SPFA as a model in its own rights. In order to provide a basis for this, the properties and the utility of SPFA factor score predictors and the possibility to identify single-item indicators in SPFA loading matrices were investigated. Regarding the factor score predictors, the main result is that the best linear predictor of the score predictor factor analysis has not only perfect determinacy but is also correlation preserving. Regarding the SPFA loadings it was found in a simulation study that five or more population factors that are represented by only one variable with a rather substantial loading can more accurately be identified by means of SPFA than with conventional factor analysis. Moreover, the percentage of correctly identified single-item indicators was substantially larger for SPFA than for the common factor model. It is therefore argued that SPFA is a tool that can be especially helpful when very short scales or single-item indicators are to be identified.

NMR spectral datasets, especially in systems with limited samples, can be difficult to interpret if they contain multiple chemical components (phases, polymorphs, molecules, crystals, glasses, etc...) and the possibility of overlapping resonances. In this paper, we benchmark several blind source separation techniques for analysis of NMR spectral datasets containing negative intensity. For benchmarking purposes, we generated a large synthetic datasbase of quadrupolar solid-state NMR-like spectra that model spin-lattice T1 relaxation or nutation tip/flip angle experiments. Our benchmarking approach focused exclusively on the ability of blind source separation techniques to reproduce the spectra of the underlying pure components. In general, we find that FastICA (Fast Independent Component Analysis), SIMPLISMA (SIMPLe-to-use-Interactive Self-modeling Mixture Analysis), and NNMF (Non-Negative Matrix Factorization) are top-performing techniques. We demonstrate that dataset normalization approaches prior to blind source separation do not considerably improve outcomes. Within the range of noise levels studied, we did not find drastic changes to the ranking of techniques. The accuracy of FastICA and SIMPLISMA degrades quickly if excess (unreal) pure components are predicted. Our results indicate poor performance of SVD (Singular Value Decomposition) methods, and we propose alternative techniques for matrix initialization. The benchmarked techniques are also applied to real solid state NMR datasets. In general, the recommendations from the synthetic datasets agree with the recommendations and results from the real data analysis. The discussion provides some additional recommendations for spectroscopists applying blind source separation to NMR datasets, and for future benchmark studies., Comment: 28 pages, 6 figures, 5 tables