Search

Principal component analysis continues to be a powerful tool in dimension reduction of high dimensional data. We assume a variance-diverging model and use the high-dimension, low-sample-size asymptotics to show that even though the principal component directions are not consistent, the sample and prediction principal component scores can be useful in revealing the population structure. We further show that these scores are biased, and the bias is asymptotically decomposed into rotation and scaling parts. We propose methods of bias-adjustment that are shown to be consistent and work well in the high dimensional situations with small sample sizes. The potential advantage of bias-adjustment is demonstrated in a classification setting.

We investigate geodesic projections of von Mises–Fisher (vMF) distributed directional data. The vMF distribution for random directions on the (p−1)-dimensional unit hypersphere Sp−1⊂Rp plays the role of multivariate normal distribution in directional statistics. For one-dimensional circle S1, the vMF distribution is called von Mises (vM) distribution. Projections onto geodesics are one of main ingredients of modeling and exploring directional data. We show that the projection of vMF distributed random directions onto any geodesic is approximately vM-distributed, albeit not exactly the same. In particular, the distribution of the geodesic-projected score is an infinite scale mixture of vM distributions. Approximations by vM distributions are given along various asymptotic scenarios including large and small concentrations (κ→∞,κ→0), high-dimensions (p→∞), and two important cases of double-asymptotics (p,κ→∞, κ∕p→c or κ∕p→λ), to support our claim: geodesic projections of the vMF are approximately vM. As one of potential applications of the result, we contemplate a projection pursuit exploration of high-dimensional directional data. We show that in a high dimensional model almost all geodesic-projections of directional data are nearly vM, thus measures of non-vM-ness are a viable candidate for projection index.

Dimension reduction of multivariate data supervised by auxiliary information is considered. A series of basis for dimension reduction is obtained as minimizers of a novel criterion. The proposed method is akin to continuum regression, and the resulting basis is called continuum directions. With a presence of binary supervision data, these directions continuously bridge the principal component, mean difference and linear discriminant directions, thus ranging from unsupervised to fully supervised dimension reduction. High-dimensional asymptotic studies of continuum directions for binary supervision reveal several interesting facts. The conditions under which the sample continuum directions are inconsistent, but their classification performance is good, are specified. While the proposed method can be directly used for binary and multi-category classification, its generalizations to incorporate any form of auxiliary data are also presented. The proposed method enjoys fast computation, and the performance is better or on par with more computer-intensive alternatives.

We consider the problem of how many components to retain in the application of principal component analysis when the dimension is much higher than the number of observations. To estimate the number of components, we propose to sequentially test skewness of the squared lengths of residual scores that are obtained by removing leading principal components. The residual lengths are asymptotically left-skewed if all principal components with diverging variances are removed, and right-skewed if not. The proposed estimator is shown to be consistent, performs well in high-dimensional simulation studies, and provides reasonable estimates in a number of real data examples.

Jung et al. (2015) introduced a geometric structure on ${\rm Sym}^+(p)$, the set of $p \times p$ symmetric positive-definite matrices, based on eigen-decomposition. Eigenstructure determines both a stratification of ${\rm Sym}^+(p)$, defined by eigenvalue multiplicities, and fibers of the "eigen-composition" map $F:M(p):=SO(p)\times{\rm Diag}^+(p)\to{\rm Sym}^+(p)$. When $M(p)$ is equipped with a suitable Riemannian metric, the fiber structure leads to notions of scaling-rotation distance between $X,Y\in {\rm Sym}^+(p)$, the distance in $M(p)$ between fibers $F^{-1}(X)$ and $F^{-1}(Y)$, and minimal smooth scaling-rotation (MSSR) curves, images in ${\rm Sym}^+(p)$ of minimal-length geodesics connecting two fibers. In this paper we study the geometry of the triple $(M(p),F,{\rm Sym}^+(p))$, focusing on some basic questions: For which $X,Y$ is there a unique MSSR curve from $X$ to $Y$? More generally, what is the set ${\cal M}(X,Y)$ of MSSR curves from $X$ to $Y$? This set is influenced by two potential types of non-uniqueness. We translate the question of whether the second type can occur into a question about the geometry of Grassmannians $G_m({\bf R}^p)$, with $m$ even, that we answer for $p\leq 4$ and $p\geq 11$. Our method of proof also yields an interesting half-angle formula concerning principal angles between subspaces of ${\bf R}^p$ whose dimensions may or may not be equal. The general-$p$ results concerning MSSR curves and scaling-rotation distance that we establish here underpin the explicit $p=3$ results in Groisser et al. (2017). Addressing the uniqueness-related questions requires a thorough understanding of the fiber structure of $M(p)$, which we also provide., arXiv admin note: text overlap with arXiv:1602.01187

A test of modality of rotationally symmetric distributions on hyperspheres is proposed. The test is based on a modified multivariate kurtosis defined for directional data on S d . We first reveal a relationship between the multivariate kurtosis and the types of modality for Euclidean data. In particular, the kurtosis of a rotationally symmetric distribution with decreasing sectional density is greater than the kurtosis of the uniform distribution, while the kurtosis of that with increasing sectional density is less. For directional data, we show an asymptotic normality of the modified spherical kurtosis, based on which a large-sample test is proposed. The proposed test of modality is applied to the problem of preventing overfitting in non-geodesic dimension reduction of directional data. The proposed test is superior than existing options in terms of computation times, accuracy and preventing overfitting. This is highlighted by a simulation study and two real data examples.

We propose a new algorithm for sparse estimation of eigenvectors in generalized eigenvalue problems (GEP). The GEP arises in a number of modern data-analytic situations and statistical methods, including principal component analysis (PCA), multiclass linear discriminant analysis (LDA), canonical correlation analysis (CCA), sufficient dimension reduction (SDR) and invariant co-ordinate selection. We propose to modify the standard generalized orthogonal iteration with a sparsity-inducing penalty for the eigenvectors. To achieve this goal, we generalize the equation-solving step of orthogonal iteration to a penalized convex optimization problem. The resulting algorithm, called penalized orthogonal iteration, provides accurate estimation of the true eigenspace, when it is sparse. Also proposed is a computationally more efficient alternative, which works well for PCA and LDA problems. Numerical studies reveal that the proposed algorithms are competitive, and that our tuning procedure works well. We demonstrate applications of the proposed algorithm to obtain sparse estimates for PCA, multiclass LDA, CCA and SDR. Supplementary materials are available online.

We investigate a geometric computational framework, called the "scaling-rotation framework", on ${\rm Sym}^+(p)$, the set of $p \times p$ symmetric positive-definite (SPD) matrices. The purpose of our study is to lay geometric foundations for statistical analysis of SPD matrices, in situations in which eigenstructure is of fundamental importance, for example diffusion-tensor imaging (DTI). Eigen-decomposition, upon which the scaling-rotation framework is based, determines both a stratification of ${\rm Sym}^+(p)$, defined by eigenvalue multiplicities, and fibers of the "eigen-composition" map $SO(p)\times{\rm Diag}^+(p)\to{\rm Sym}^+(p)$. This leads to the notion of scaling-rotation distance [Jung et al. (2015)], a measure of the minimal amount of scaling and rotation needed to transform an SPD matrix, $X,$ into another, $Y,$ by a smooth curve in ${\rm Sym}^+(p)$. Our main goal in this paper is the systematic characterization and analysis of minimal smooth scaling-rotation (MSSR) curves, images in ${\rm Sym}^+(p)$ of minimal-length geodesics connecting two fibers in the "upstairs" space $SO(p)\times{\rm Diag}^+(p)$. The length of such a geodesic connecting the fibers over $X$ and $Y$ is what we define to be the scaling-rotation distance from $X$ to $Y.$ For the important low-dimensional case $p = 3$ (the home of DTI), we find new explicit formulas for MSSR curves and for the scaling-rotation distance, and identify ${\cal M}(X,Y)$ in all "nontrivial" cases. The quaternionic representation of $SO(3)$ is used in these computations. We also provide closed-form expressions for scaling-rotation distance and MSSR curves for the case $p = 2$., 15 figures

This paper analyzes the linear degrees of freedom (LDoF) for K-user M ×N MIMO interference channels with constant channel coefficients. In this correspondence, we interpret the interference alignment problem as a multivariate polynomial system in terms of properness. The properness provides a LDoF upperbound for interference alignment. With this approach, we propose a method to obtain the upperbound for total LDoF achievable by linear interference alignment. The LDoF upperbound for properness condition results have two different cases which depend on the number of users, the number of transmit antennas, and the number of receive antennas. A key contribution of this paper is that it provides a closed-form LDoF upperbound for M ×N MIMO interference channels where the LDoF of each user may be different.

The multiuser MIMO (MU-MIMO) systems tend to have inter-user interference in practical systems with limited feedback. Practical MU-MIMO systems with limited feedback can be divided into two classes in general. One is zero-forcing beamforming (ZFBF), and the other is per user unitary rate control (PU2RC) which is an extended version of random beamforming (RBF). To improve performance, ZFBF needs more feedback bits, and PU2RC needs a larger number of users. We propose a new hybrid MU-MISO system which has the advantages of the two MU-MIMO schemes simultaneously, and analyze the sum-rate performance and the quantization error of the hybrid scheme. The analysis shows how the number of feedback bits, the number of users, and the signal power affect the sum-rate. The throughput scaling laws are also derived in the high and the medium SNR regimes. Simulation results show that the proposed hybrid MU-MISO system performs better than either of the two MU-MIMO systems with the same number of feedback bits. Another advantage is that it has lower complexity for codeword search since it has a hierarchical codebook structure.

This paper presents a new and simple decoding algorithm for layered space time block codes such as the two independent Alamouti's codes which are also called the double space-time transmit diversity (DSTTD) system. By using group interference suppression and successive interference cancellation, we can treat DSTTD as two independent space-time block codes (STBC). We can then decode both of these STBC's through a simple maximum likelihood (ML) detector with null space-based interference cancellation. We also compare the proposed interference cancellation (IC) scheme with the conventional MMSE IC scheme. The performance of the proposed IC scheme is comparable to that of the MMSE IC scheme while the complexity reduction factor of the proposed scheme can be up to 5 compared to the MMSE IC scheme.

We present a novel method that adaptively deforms a polysphere (a product of spheres) into a single high dimensional sphere which then allows for principal nested spheres (PNS) analysis. Applying our method to skeletal representations of simulated bodies as well as of data from real human hippocampi yields promising results in view of dimension reduction. Specifically in comparison to composite PNS (CPNS), our method of principal nested deformed spheres (PNDS) captures essential modes of variation by lower dimensional representations.

We introduce a new geometric framework for the set of symmetric positive-definite (SPD) matrices, aimed to characterize deformations of SPD matrices by individual scaling of eigenvalues and rotation of eigenvectors of the SPD matrices. To characterize the deformation, the eigenvalue-eigenvector decomposition is used to find alternative representations of SPD matrices, and to form a Riemannian manifold so that scaling and rotations of SPD matrices are captured by geodesics on this manifold. The problems of non-unique eigen-decompositions and eigenvalue multiplicities are addressed by finding minimal-length geodesics, which gives rise to a distance and an interpolation method for SPD matrices. Computational procedures to evaluate the minimal scaling--rotation deformations and distances are provided for the most useful cases of $2 \times 2$ and $3 \times 3$ SPD matrices. In the new geometric framework, minimal scaling--rotation curves interpolate eigenvalues at constant logarithmic rate, and eigenvectors at constant angular rate. In the context of diffusion tensor imaging, this results in better behavior of the trace, determinant and fractional anisotropy of interpolated SPD matrices in typical cases.

A general framework for a novel non-geodesic decomposition of high-dimensional spheres or high-dimensional shape spaces for planar landmarks is discussed. The decomposition, principal nested spheres, leads to a sequence of submanifolds with decreasing intrinsic dimensions, which can be interpreted as an analogue of principal component analysis. In a number of real datasets, an apparent one-dimensional mode of variation curving through more than one geodesic component is captured in the one-dimensional component of principal nested spheres. While analysis of principal nested spheres provides an intuitive and flexible decomposition of the high-dimensional sphere, an interesting special case of the analysis results in finding principal geodesics, similar to those from previous approaches to manifold principal component analysis. An adaptation of our method to Kendall's shape space is discussed, and a computational algorithm for fitting principal nested spheres is proposed. The result provides a coordinate system to visualize the data structure and an intuitive summary of principal modes of variation, as exemplified by several datasets. Copyright 2012, Oxford University Press.

In High Dimension, Low Sample Size (HDLSS) data situations, where the dimension d is much larger than the sample size n, principal component analysis (PCA) plays an important role in statistical analysis. Under which conditions does the sample PCA well reflect the population covariance structure? We answer this question in a relevant asymptotic context where d grows and n is fixed, under a generalized spiked covariance model. Specifically, we assume the largest population eigenvalues to be of the order dα, where α1. Earlier results show the conditions for consistency and strong inconsistency of eigenvectors of the sample covariance matrix. In the boundary case, α=1, where the sample PC directions are neither consistent nor strongly inconsistent, we show that eigenvalues and eigenvectors do not degenerate but have limiting distributions. The result smoothly bridges the phase transition represented by the other two cases, and thus gives a spectrum of limits for the sample PCA in the HDLSS asymptotics. While the results hold under a general situation, the limiting distributions under Gaussian assumption are illustrated in greater detail. In addition, the geometric representation of HDLSS data is extended to give three different representations, that depend on the magnitude of variances in the first few principal components.

This paper analyzes the optimal degrees of freedom (DoF) for K-user M × N MIMO interference channels with constant channel coefficients. In this paper, we use the methodology of interpreting the interference alignment problem as a multivariate polynomial system. The properness of the system enables us to determine the feasibility of interference alignment. Using this methodology, we propose an algorithm to obtain the optimal DoF, and compute the closed-form optimal DoF. We show that K min(M,N) is optimal DoF if δ ≥ min(M,N), where δ = ⌊M+N over K+1⌋. If δ < min(M,N), K M+N over K+1 is optimal DoF when M+N over K+1 is an integer, and ⌈K M+N over K+1⌉ − 1 is the optimal DoF when M+N over K+1 is not an integer. A key contribution of this paper is that it provides a closed-form optimal DoF for M × N MIMO interference channels when the DoF of each user can be different.

A generalized Principal Component Analysis (PCA) for manifold-valued shapes is discussed with forward and backward stepwise views of PCA. Finite and infinite dimensional shape spaces are briefly introduced. A backward extension of PCA for the shape spaces, called Principal Nested Spheres, is shown in more detail, which results in a non-geodesic decomposition of shape spaces, capturing more variation in lower dimension.

This paper presents closed-form expressions of the distribution of signal-to-interference-plus-noise ratios (SINRs) and the average bit error rate (BER) for an orthogonal random beamforming system in multiple-antenna Gaussian broadcast channels. First, we derive the statistical distribution of the SINR of the scheduled transmit signals and confirm that the approximate form of the cumulative distribution function (cdf) of the SINR is equivalent to the formula in \cite{sharif} and \cite{letaief}. We then obtain a closed-form expression for the total average BER of the system by using the approximate cdf. Our theoretical results are compared to the Monte Carlo simulation results.

Principal component analysis (PCA) for various types of image data is analyzed in terms of the forward and backward stepwise viewpoints. In the traditional forward view, PCA and approximating subspaces are constructed from lower dimension to higher dimension. The backward approach builds PCA in the reverse order from higher dimension to lower dimension.We see that for manifold data the backward view gives much more natural and accessible generalizations of PCA. As a backward stepwise approach, composite Principal Nested Spheres, which generalizes PCA, is proposed. In an example describing the motion of the lung based on CT images, we show that composite Principal Nested Spheres captures landmark data more succinctly than forward PCA methods.

Principal Component Analysis (PCA) is an important tool of dimension reduction especially when the dimension (or the number of variables) is very high. Asymptotic studies where the sample size is fixed, and the dimension grows [i.e., High Dimension, Low Sample Size (HDLSS)] are becoming increasingly relevant. We investigate the asymptotic behavior of the Principal Component (PC) directions. HDLSS asymptotics are used to study consistency, strong inconsistency and subspace consistency. We show that if the first few eigenvalues of a population covariance matrix are large enough compared to the others, then the corresponding estimated PC directions are consistent or converge to the appropriate subspace (subspace consistency) and most other PC directions are strongly inconsistent. Broad sets of sufficient conditions for each of these cases are specified and the main theorem gives a catalogue of possible combinations. In preparation for these results, we show that the geometric representation of HDLSS data holds under general conditions, which includes a $\rho$-mixing condition and a broad range of sphericity measures of the covariance matrix., Comment: Published in at http://dx.doi.org/10.1214/09-AOS709 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

This paper presents a new and simple decoding algorithm for layered space time block codes such as the two independent Alamouti's codes which are also called the double space-time transmit diversity (DSTTD) system. By using group interference suppression and successive interference cancellation, we can treat DSTTD as two independent space-time block codes (STBC). We can then decode both of these STBC's through a simple maximum likelihood (ML) detector with null space-based interference cancellation. We also compare the proposed interference cancellation (IC) scheme with the conventional MMSE IC scheme. The performance of the proposed IC scheme is comparable to that of the MMSE IC scheme while the complexity of the proposed scheme is lower than that of the MMSE IC scheme.