Search

Sufficient dimension reduction methods often require stringent conditions on the joint distribution of the predictor, or, when such conditions are not satisfied, rely on marginal transformation or reweighting to fulfill them approximately. For example, a typical dimension reduction method would require the predictor to have elliptical or even multivariate normal distribution. In this paper, we reformulate the commonly used dimension reduction methods, via the notion of "central solution space," so as to circumvent the requirements of such strong assumptions, while at the same time preserve the desirable properties of the classical methods, such as $\sqrt{n}$-consistency and asymptotic normality. Imposing elliptical distributions or even stronger assumptions on predictors is often considered as the necessary tradeoff for overcoming the "curse of dimensionality," but the development of this paper shows that this need not be the case. The new methods will be compared with existing methods by simulation and applied to a data set., Comment: Published in at http://dx.doi.org/10.1214/08-AOS598 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

The paper presents a unified jackknife theory for a fairly general class of mixed models which includes some of the widely used mixed linear models and generalized linear mixed models as special cases. The paper develops jackknife theory for the important, but so far neglected, prediction problem for the general mixed model. For estimation of fixed parameters, a jackknife method is considered for a general class of M-estimators which includes the maximum likelihood, residual maximum likelihood and ANOVA estimators for mixed linear models and the recently developed method of simulated moments estimators for generalized linear mixed models. For both the prediction and estimation problems, a jackknife method is used to obtain estimators of the mean squared errors (MSE). Asymptotic unbiasedness of the MSE estimators is shown to hold essentially under certain moment conditions. Simulation studies undertaken support our theoretical results.

When using nonparametric estimates of the mean curve, surface or image underlying noisy observations, the selection of "smoothing parameters" is generally crucial. This paper gives a theoretical comparison of the performances of generalized cross-validation (GCV) and of its fast randomized version (RGCV), as selection criteria. This is mainly done by studying the asymptotic distribution of the excess error for each selector, that is, the difference between the (data-driven) resulting average squared error (ASE) and the best possible ASE. We show here that, by using randomization, this distribution is dilated, as compared to that for CV or GCV, only by a factor always lower than $1 + 1/n_R$, where $n_R$ is the number of primary randomized trace estimates one uses in RGCV. We include in the compared selectors, the partial cross-validation (PCV) approach where only a fraction of all the possible "leave-one-out" validation tests are evaluated; so that PCV is a common practice to reduce the computational cost in many contexts. In this paper, PCV will in fact appear as quite inefficient as compared to RGCV from this computational point of view. Moreover, we show that a precise comparison (and interpretation of the gain of using $n_R \geq 2$) is possible in terms of equivalent (in distribution) excess errors, if PCV uses a certain percentage of the test points greater than 50%. The obtained comparisons will be seen as quite reassuring on what is "sacrificed" in using randomized selectors. We give rigorous results mainly for the kernel regression setting as in the previous detailed study by Hardle, Hall and Marron of standard selectors, except that we do not restrict this one to an equidistant design.

Existing frequency domain methods for bootstrapping time series have a limited range. Essentially, these procedures cover the case of linear time series with independent innovations, and some even require the time series to be Gaussian. In this paper we propose a new frequency domain bootstrap method—the hybrid periodogram bootstrap (HPB)—which is consistent for a much wider range of stationary, even nonlinear, processes and which can be applied to a large class of periodogram-based statistics. The HPB is designed to combine desirable features of different frequency domain techniques while overcoming their respective limitations. It is capable to imitate the weak dependence structure of the periodogram by invoking the concept of convolved subsampling in a novel way that is tailor-made for periodograms. We show consistency for the HPB procedure for a general class of stationary time series, ranging clearly beyond linear processes, and for spectral means and ratio statistics on which we mainly focus. The finite sample performance of the new bootstrap procedure is illustrated via simulations.

Sliced inverse regression (SIR) is an innovative and effective method for sufficient dimension reduction and data visualization. Recently, an impressive range of penalized SIR methods has been proposed to estimate the central subspace in a sparse fashion. Nonetheless, few of them considered the sparse sufficient dimension reduction from a decision-theoretic point of view. To address this issue, we in this paper establish the minimax rates of convergence for estimating the sparse SIR directions under various commonly used loss functions in the literature of sufficient dimension reduction. We also discover the possible trade-off between statistical guarantee and computational performance for sparse SIR. We finally propose an adaptive estimation scheme for sparse SIR which is computationally tractable and rate optimal. Numerical studies are carried out to confirm the theoretical properties of our proposed methods.

Several new methods have been recently proposed for performing valid inference after model selection. An older method is sample splitting: use part of the data for model selection and the rest for inference. In this paper, we revisit sample splitting combined with the bootstrap (or the Normal approximation). We show that this leads to a simple, assumption-lean approach to inference and we establish results on the accuracy of the method. In fact, we find new bounds on the accuracy of the bootstrap and the Normal approximation for general nonlinear parameters with increasing dimension which we then use to assess the accuracy of regression inference. We define new parameters that measure variable importance and that can be inferred with greater accuracy than the usual regression coefficients. Finally, we elucidate an inference-prediction trade-off: splitting increases the accuracy and robustness of inference but can decrease the accuracy of the predictions.

The paper focuses on the problem of model selection in linear Gaussian regression with unknown possibly inhomogeneous noise. For a given family of linear estimators $\{\widetilde{\boldsymbol{{\theta}}}_{m},m\in\mathscr{M}\}$, ordered by their variance, we offer a new “smallest accepted” approach motivated by Lepski’s device and the multiple testing idea. The procedure selects the smallest model which satisfies the acceptance rule based on comparison with all larger models. The method is completely data-driven and does not use any prior information about the variance structure of the noise: its parameters are adjusted to the underlying possibly heterogeneous noise by the so-called “propagation condition” using a wild bootstrap method. The validity of the bootstrap calibration is proved for finite samples with an explicit error bound. We provide a comprehensive theoretical study of the method, describe in details the set of possible values of the selected model $\widehat{m}\in\mathscr{M}$ and establish some oracle error bounds for the corresponding estimator $\widehat{\boldsymbol{{\theta}}}=\widetilde{\boldsymbol{{\theta}}}_{\widehat{m}}$.

We extend the available asymptotic theory for autoregressive sieve estimators to cover the case of stationary and invertible linear processes driven by independent identically distributed (i.i.d.) infinite variance (IV) innovations. We show that the ordinary least squares sieve estimates, together with estimates of the impulse responses derived from these, obtained from an autoregression whose order is an increasing function of the sample size, are consistent and exhibit asymptotic properties analogous to those which obtain for a finite-order autoregressive process driven by i.i.d. IV errors. As these limit distributions cannot be directly employed for inference because they either may not exist or, where they do, depend on unknown parameters, a second contribution of the paper is to investigate the usefulness of bootstrap methods in this setting. Focusing on three sieve bootstraps: the wild and permutation bootstraps, and a hybrid of the two, we show that, in contrast to the case of finite variance innovations, the wild bootstrap requires an infeasible correction to be consistent, whereas the other two bootstrap schemes are shown to be consistent (the hybrid for symmetrically distributed innovations) under general conditions.

Multivariate time series present many challenges, especially when they are high dimensional. The paper's focus is twofold. First, we address the subject of consistently estimating the autocovariance sequence; this is a sequence of matrices that we conveniently stack into one huge matrix. We are then able to show consistency of an estimator based on the so-called flat-top tapers; most importantly, the consistency holds true even when the time series dimension is allowed to increase with the sample size. Second, we revisit the linear process bootstrap (LPB) procedure proposed by McMurry and Politis [J. Time Series Anal. 31 (2010) 471-482] for univariate time series. Based on the aforementioned stacked autocovariance matrix estimator, we are able to define a version of the LPB that is valid for multivariate time series. Under rather general assumptions, we show that our multivariate linear process bootstrap (MLPB) has asymptotic validity for the sample mean in two important cases: (a) when the time series dimension is fixed and (b) when it is allowed to increase with sample size. As an aside, in case (a) we show that the MLPB works also for spectral density estimators which is a novel result even in the univariate case. We conclude with a simulation study that demonstrates the superiority of the MLPB in some important cases., Published at http://dx.doi.org/10.1214/14-AOS1301 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

The Gaussian graphical model, a popular paradigm for studying relationship among variables in a wide range of applications, has attracted great attention in recent years. This paper considers a fundamental question: When is it possible to estimate low-dimensional parameters at parametric square-root rate in a large Gaussian graphical model? A novel regression approach is proposed to obtain asymptotically efficient estimation of each entry of a precision matrix under a sparseness condition relative to the sample size. When the precision matrix is not sufficiently sparse, or equivalently the sample size is not sufficiently large, a lower bound is established to show that it is no longer possible to achieve the parametric rate in the estimation of each entry. This lower bound result, which provides an answer to the delicate sample size question, is established with a novel construction of a subset of sparse precision matrices in an application of Le Cam's lemma. Moreover, the proposed estimator is proven to have optimal convergence rate when the parametric rate cannot be achieved, under a minimal sample requirement. The proposed estimator is applied to test the presence of an edge in the Gaussian graphical model or to recover the support of the entire model, to obtain adaptive rate-optimal estimation of the entire precision matrix as measured by the matrix $\ell_q$ operator norm and to make inference in latent variables in the graphical model. All of this is achieved under a sparsity condition on the precision matrix and a side condition on the range of its spectrum. This significantly relaxes the commonly imposed uniform signal strength condition on the precision matrix, irrepresentability condition on the Hessian tensor operator of the covariance matrix or the $\ell_1$ constraint on the precision matrix. Numerical results confirm our theoretical findings. The ROC curve of the proposed algorithm, Asymptotic Normal Thresholding (ANT), for support recovery significantly outperforms that of the popular GLasso algorithm., Comment: Published at http://dx.doi.org/10.1214/14-AOS1286 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

This paper formulates a penalized empirical likelihood (PEL) method for inference on the population mean when the dimension of the observations may grow faster than the sample size. Asymptotic distributions of the PEL ratio statistic is derived under different component-wise dependence structures of the observations, namely, (i) non-Ergodic, (ii) long-range dependence and (iii) short-range dependence. It follows that the limit distribution of the proposed PEL ratio statistic can vary widely depending on the correlation structure, and it is typically different from the usual chi-squared limit of the empirical likelihood ratio statistic in the fixed and finite dimensional case. A unified subsampling based calibration is proposed, and its validity is established in all three cases, (i)-(iii). Finite sample properties of the method are investigated through a simulation study., Published in at http://dx.doi.org/10.1214/12-AOS1040 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

International audience; In the context of correlated multiple tests, we aim to nonasymptotically control the family-wise error rate (FWER) using resampling-type procedures. We observe repeated realizations of a Gaussian random vector in possibly high dimension and with an unknown covariance matrix, and consider the one- and two-sided multiple testing problem for the mean values of its coordinates. We address this problem by using the confidence regions developed in the companion paper [Ann. Statist. 38 (2010) 51-82], which lead directly to single-step procedures; these can then be improved using step-down algorithms, following an established general methodology laid down by Romano and Wolf [J. Amer. Statist. Assoc. 100 (2005) 94–108]. This gives rise to several different procedures, whose performances are compared using simulated data.

A low-degree polynomial model for a response curve is used commonly in practice. It generally incorporates a linear or quadratic function of the covariate. In this paper we suggest methods for testing the goodness of fit of a general polynomial model when there are errors in the covariates. There, the true covariates are not directly observed, and conventional bootstrap methods for testing are not applicable. We develop a new approach, in which deconvolution methods are used to estimate the distribution of the covariates under the null hypothesis, and a ``wild'' or moment-matching bootstrap argument is employed to estimate the distribution of the experimental errors (distinct from the distribution of the errors in covariates). Most of our attention is directed at the case where the distribution of the errors in covariates is known, although we also discuss methods for estimation and testing when the covariate error distribution is estimated. No assumptions are made about the distribution of experimental error, and, in particular, we depart substantially from conventional parametric models for errors-in-variables problems., Published in at http://dx.doi.org/10.1214/009053607000000361 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

This paper introduces a version of empirical likelihood based on the periodogram and spectral estimating equations. This formulation handles dependent data through a data transformation (i.e., a Fourier transform) and is developed in terms of the spectral distribution rather than a time domain probability distribution. The asymptotic properties of frequency domain empirical likelihood are studied for linear time processes exhibiting both short- and long-range dependence. The method results in likelihood ratios which can be used to build nonparametric, asymptotically correct confidence regions for a class of normalized (or ratio) spectral parameters, including autocorrelations. Maximum empirical likelihood estimators are possible, as well as tests of spectral moment conditions. The methodology can be applied to several inference problems such as Whittle estimation and goodness-of-fit testing., Comment: Published at http://dx.doi.org/10.1214/009053606000000902 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

In this paper we consider the problem of bootstrapping a class of spatial regression models when the sampling sites are generated by a (possibly nonuniform) stochastic design and are irregularly spaced. It is shown that the natural extension of the existing block bootstrap methods for grid spatial data does not work for irregularly spaced spatial data under nonuniform stochastic designs. A variant of the blocking mechanism is proposed. It is shown that the proposed block bootstrap method provides a valid approximation to the distribution of a class of M-estimators of the spatial regression parameters. Finite sample properties of the method are investigated through a moderately large simulation study and a real data example is given to illustrate the methodology., Published at http://dx.doi.org/10.1214/009053606000000551 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Resampling methods are frequently used in practice to adjust critical values of nonparametric tests. In the present paper a comprehensive and unified approach for the conditional and unconditional analysis of linear resampling statistics is presented. Under fairly mild assumptions we prove tightness and an asymptotic series representation for their weak accumulation points. From this series it becomes clear which part of the resampling statistic is responsible for asymptotic normality. The results leads to a discussion of the asymptotic correctness of resampling methods as well as their applications in testing hypotheses. They are conditionally correct iff a central limit theorem holds for the original test statistic. We prove unconditional correctness iff the central limit theorem holds or when symmetric random variables are resampled by a scheme of asymptotically random signs. Special cases are the m (n) out of k (n) bootstrap, the weighted bootstrap, the wild bootstrap and all kinds of permutation statistics. The program is carried out for convergent partial sums of rowwise independent infinitesimal triangular arrays in detail. These results are used to compare power functions of conditional resampling tests and their unconditional counterparts. The proof uses the method of random scores for permutation type statistics.

Usually, when testing the null hypothesis that a distribution has one mode against the alternative that it has two, the null hypothesis is interpreted as entailing that the density of the sampling distribution has a unique point of zero slope, which is a local maximum. In this paper we argue that a more appropriate null hypothesis is that the density has two points of zero slope, of which one is a local maximum and the other is a shoulder. We show that when a test for a mode-with-shoulder is properly calibrated, so that it has asymptotically correct level, it is generally conservative when applied to the case of a mode without a shoulder. We suggest methods for calibrating both the bandwidth and dip-excess mass tests in the setting of a mode with a shoulder. We also provide evidence in support of the converse: a test calibrated for a single mode without a shoulder tends to be anticonservative when applied to a mode with a shoulder. The calibration method involves resampling from a ‘‘template’’ density with exactly one mode and one shoulder. It exploits the following asymptotic factorization property for both the sample and resample forms of the test statistic: all dependence of these quantities on the sampling distribution cancels asymptotically from their ratio. In contrast to other approaches, the method has very good adaptivity properties.

In this paper the validity of a one-term Edgeworth expansion for Studentized symmetric statistics is proved. We propose jackknife estimates for the unknown constants appearing in the expansion and prove their consistency. As a result we obtain the second-order correctness of the empirical Edgeworth expansion for a very general class of statistics, including $U$-statistics, $L$-statistics and smooth functions of the sample mean. We illustrate the application of the bootstrap in the case of a $U$-statistic of degree two.

In this paper, we study the weak convergence to an appropriate Gaussian process of the empirical process of the block-based bootstrap estimator proposed by Kunsch for stationary sequences. The classes of processes investigated are weak dependent and associated sequences. We also prove that, differently from the independent situation, the bootstrapped estimator of the mean of certain dependent sequences satisfies the central limit theorem while the mean of the original sequence does not.

Applications of extreme value theory to problems of statistical inference typically involve estimating tail probabilities well beyond the range of the data, without the benefit of a concise mathematical model for the sampling distribution. The available model is generally only an asymptotic one. That is, an approximation to probabilities of extreme deviation is supposed, which is assumed to become increasingly accurate as one moves further from the range of the data, but whose concise accuracy is unknown. Quantification of the level of accuracy is essential for optimal estimation of tail probabilities. In the present paper we suggest a practical device, based on a nonstandard application of the bootstrap, for determining empirically the accuracy of the approximation and thereby constructing appropriate estimators.

Let $\hat{F}_n$ be the Kaplan-Meier estimator of a distribution function F computed from randomly censored data. It is known that, under certain integrability assumptions on a function $\varphi$, the Kaplan-Meier integral $\int \varphi d \hat{F}_n$, when properly standardized, is asymptotically normal. In this paper it is shown that, with probability 1, the jackknife estimate of variance consistently estimates the (limit) variance.