Search

Often a problem in linear regression either for correlated data or high-dimensional data is the singularity of the sample covariance matrix. While dealing with an ill-conditioned covariance matrix has been a longstanding challenge in the statistical literature, recent proposals relying on adding noises to the original data have found their successes in obtaining reliable estimates and making predictions. It has been shown that perturbing data with noises has a close relationship to the well-known ridge estimator. In this paper, we propose to add noises to the predictors with a known covariance structure, and call the estimator obtained in such a way a “pseudo estimator”. A new variable selection procedure based on the concept of pseudo confidence interval which works for both correlated predictors and “large p small n ” problem is proposed. We study the theoretical properties of the proposed pseudo estimator and variable selection procedure. Furthermore, we use an ensemble step to stabilize the pseudo estimation (variable selection) results. The advantages of the proposed method are demonstrated by both simulation studies and real data analyses.

–Many well-known sufficient dimension reduction methods investigate the inverse conditional moments of the predictors given the response. The required linearity condition, the number and ar...

We propose two sufficient variable selection procedures, i.e., one- and two-stage approaches using independence measures for continuous response, illustrated by distance correlation and the Hilbert–Schmidt Independence Criterion correlation. We show the advantages of the proposed procedures over some existing marginal screening methods through simulations and a real data analysis. Our procedures are model-free and thus robust against model mis-specification. They are particularly useful when some active predictors are marginally independent of the response.

We propose a novel class of independence measures for testing independence between two random vectors based on the discrepancy between the conditional and the marginal characteristic functions. The...

In this article, we propose a new method for sufficient dimension reduction when both response and predictor are vectors. The new method, using distance covariance, keeps the model-free advantage, ...

Through investigating a recently introduced sufficient dimension reduction method with Hellinger index, this article shows that the generalized Hellinger index unifies three existing dimension reduction methods: kernel discriminant analysis, sliced regression and density minimum average variance estimation, with certain weight functions. The Hellinger index is then extended to regression models with multivariate responses. Furthermore, new algorithms based on Hellinger index to estimate the dual central subspaces and to enable variable selection for sparse models are proposed. Simulation studies and a real data analysis demonstrate the efficacy of the proposed approaches.

In this paper, we study dimension folding for matrix/array structured predictors with categorical variables. The categorical variable information is incorporated into dimension folding for regression and classification. The concepts of marginal, conditional, and partial folding subspaces are introduced, and their connections to central folding subspace are investigated. Three estimation methods are proposed to estimate the desired partial folding subspace. An empirical maximal eigenvalue ratio criterion is used to determine the structural dimensions of the associated partial folding subspace. Effectiveness of the proposed methods is evaluated through simulation studies and an application to a longitudinal data.

Estimating an inverse regression space is especially important in sufficient dimension reduction. However, it typically requires a tuning parameter, such as the number of slices in a slicing method...

Sufficient dimension reduction, replacing the original predictors with a few linear combinations while keeping all the regression information, has been widely studied. A key goal is to find the central mean subspace, the intersection of all subspaces that provide such a reduction. To this end, a new sufficient dimension reduction method is proposed, with two estimation procedures, through a novel approach of feature filter, applicable to both univariate and multivariate responses. Asymptotic results are established. Estimation methods to determine the structural dimension, to obtain sparse estimator and to deal with large p small n data are provided. The efficacy of the method is demonstrated by simulations and a real data example.

By using Hilbert–Schmidt Independence Criterion, a sufficient dimension reduction method is proposed to estimate the directions in multiple-index models. A projection pursuit type of sufficient searching algorithm is introduced to reduce the computational complexity, as the original problem involves non-linear optimization over multidimensional Grassmann-manifold. A bootstrap procedure with additional jump point detection algorithm is used for determining the dimensionality. The proposed method demonstrates competitive performance compared with some well-known dimension reduction methods via simulation studies and an application to a real data.

In this article, we propose a new efficient sparse estimate (ESE) in sufficient dimension reduction using distance covariance. Our method is model-free and does not need any kernel function or slicing selection. Moreover, it can naturally deal with multivariate response scenarios, making it appealing in a modified sequential algorithm that targets the large p small n problems. Compared with screening procedures that only use marginal utility, our method can extract more useful information from the data and is capable of determining the size of the selected submodel automatically while most of screening procedures cannot. Under mild conditions, based on manifold theories and techniques, it can be shown that our method would perform asymptotically as well as if the true irrelevant predictors were known, which is referred to as the oracle property. Extensive simulation studies and two real data examples demonstrate the effectiveness and efficiency of the proposed approach. It is remarkable that the a...

Sufficient dimension reduction (SDR) is a paradigm for reducing the dimension of the predictors without losing regression information. Most SDR methods require inverting the covariance matrix of the predictors. This hinders their use in the analysis of contemporary datasets where the number of predictors exceeds the available sample size and the predictors are highly correlated. To this end, by incorporating the seeded SDR idea and the sequential dimension-reduction framework, we propose a SDR method for high-dimensional data with correlated predictors. The performance of the proposed method is studied via extensive simulations. To demonstrate its use, an application to microarray gene expression data where the response is the production rate of riboflavin (vitamin B2) is presented.

Existing dimension reduction methods in multivariate analysis have focused on reducing sets of random vectors into equivalently sized dimensions, while methods in regression settings have focused mainly on decreasing the dimension of the predictor variables. However, for problems involving a multivariate response, reducing the dimension of the response vector is also desirable and important. In this paper, we develop a new concept, termed the Dual Central Subspaces (DCS), to produce a method for simultaneously reducing the dimensions of two sets of random vectors, irrespective of the labels predictor and response. Different from previous methods based on extensions of Canonical Correlation Analysis (CCA), the recovery of this subspace provides a new research direction for multivariate sufficient dimension reduction. A particular model-free approach is detailed theoretically and the performance investigated through simulation and a real data analysis.

In this paper, we introduce sufficient dimension folding for a functional of conditional distribution of matrix- or array-valued objects, which suggests a new concept of central T dimension folding subspace (CTDFS). CTDFS includes central dimension folding subspace and central mean dimension folding subspace as special cases. A class of estimation methods on CTDFS is introduced. In particular, we focus on sufficient dimension folding for robust functionals. In this paper, we pay special attention to the central quantile dimension folding subspace (CQDFS), a widely interesting case of CTDFS, and develop new estimation methods. The performances of the proposed estimation methods on estimating the CQDFS are demonstrated by simulations and by analysing the primary biliary cirrhosis data.

Sufficient dimension reduction is a useful tool for studying the dependence between a response and a multi-dimensional predictor. In this article, a new formulation is proposed that is based on the Hellinger integral of order two, introduced as a natural measure of the regression information contained in the predictor subspace. The response may be either continuous or discrete. We establish links between local and global central subspaces, and propose an efficient local estimation algorithm. Simulations and an application show that our method compares favourably with existing approaches.

Summary We propose a new and simple framework for dimension reduction in the large p, small n setting. The framework decomposes the data into pieces, thereby enabling existing approaches for n>p to be adapted to n < p problems. Estimating a large covariance matrix, which is a very difficult task, is avoided. We propose two separate paths to implement the framework. Our paths provide sufficient procedures for identifying informative variables via a sequential approach. We illustrate the paths by using sufficient dimension reduction approaches, but the paths are very general. Empirical evidence demonstrates the efficacy of our paths. Additional simulations and applications are given in an on-line supplementary file.

In this article, we consider sufficient dimension folding for the regression mean function when predictors are matrix- or array-valued. We propose a new concept named central mean dimension folding subspace and its two local estimation methods: folded outer product of gradients estimation (folded-OPG) and folded minimum average variance estimation (folded-MAVE). We establish the asymptotic properties for folded-MAVE. A modified BIC criterion is used to determine the dimensions of the central mean dimension folding subspace. We evaluate the performances of the two local estimation methods by simulated examples and demonstrate the efficacy of folded-MAVE in finite samples. And in particular, we apply our methods to analyze a longitudinal study of primary biliary cirrhosis. Supplementary materials for this article are available online.

In the multiple linear regression setting, we propose a general framework, termed weighted orthogonal components regression (WOCR), which encompasses many known methods as special cases, including ridge regression and principal components regression. WOCR makes use of the monotonicity inherent in orthogonal components to parameterize the weight function. The formulation allows for efficient determination of tuning parameters and hence is computationally advantageous. Moreover, WOCR offers insights for deriving new better variants. Specifically, we advocate weighting components based on their correlations with the response, which leads to enhanced predictive performance. Both simulated studies and real data examples are provided to assess and illustrate the advantages of the proposed methods., 22 pages, 4 figures

In this paper we study time-varying coefficient (beta coefficient) models with a time trend function to characterize the nonlinear, non-stationary and trending phenomenon in time series and to explain the behavior of asset returns. The general local polynomial method is developed to estimate the time trend and coefficient functions. More importantly, a graphical tool, the plot of the kth-order derivative of the parameter versus time, is proposed to select the proper order of the local polynomial so that the best estimate can be obtained. Finally, we conduct Monte Carlo experiments and a real data analysis to examine the finite sample performance of the proposed modeling procedure and compare it with the Nadaraya–Watson method as well as the local linear method.

In this article, we give an alternative and much shorter proof for the invariance law of surrogate dimension reduction for measurement error regression that was established by Li and Yin (2007).

In this article, we propose an alternating inverse regression (AIR) to estimate the central subspace (CS) in a successive manner. Taking advantage of both sliced inverse regression (SIR) and partial least squares (PLS), AIR circumvents the collinearity and curse of dimensionality simultaneously. A modified BIC criterion with a penalty term achieving an optimal convergence rate is suggested to estimate the dimension of the CS. We also extend AIR to the multivariate responses case. Through illustrative examples and a real dataset, we demonstrate the usefulness of AIR, and its advantages over some existing methods. Supplemental materials for this article are available online.

The quadratic response surface model is quite effective in data analysis. One important issue is the estimation of near-stationary conditions including a ridge. When all the eigenvalues of the quadratic matrix are simple, Carter et al. [5] obtain asymptotic results to calculate their confidence intervals. Bisgaard and Ankenman [1] introduce a simple double linear regression method to obtain an equivalent result. However, their methods may be biased when there are multiple roots, an issue particularly when multiple roots are zero. Yin and Seymour [16] modify a procedure based on the result by Bisgaard and Ankenman [1] to obtain the intervals for multiple roots. In this expository article, we apply a recently developed dimension reduction method to the quadratic response surface model in estimating the dimensionality of a ridge under very general conditions.