Search

Given finite i.i.d. samples in a Hilbert space with zero mean and trace-class covariance operator Σ , the problem of recovering the spectral projectors of Σ naturally arises in many applications. In this paper, we consider the problem of finding distributional approximations of the spectral projectors of the empirical covariance operator Σ ^ , and offer a dimension-free framework where the complexity is characterized by the so-called relative rank of Σ . In this setting, novel quantitative limit theorems and bootstrap approximations are presented subject to mild conditions in terms of moments and spectral decay. In many cases, these even improve upon existing results in a Gaussian setting. [ABSTRACT FROM AUTHOR]

Estimating parameters of functional ARMA, GARCH and invertible processes requires estimating lagged covariance and cross‐covariance operators of Cartesian product Hilbert space‐valued processes. Asymptotic results have been derived in recent years, either less generally or under a strict condition. This article derives upper bounds of the estimation errors for such operators based on the mild condition Lp‐m‐approximability for each lag, Cartesian power(s) and sample size, where the two processes can take values in different spaces in the context of lagged cross‐covariance operators. Implications of our results on eigen elements and parameters in functional AR(MA) models are also discussed. [ABSTRACT FROM AUTHOR]

We consider a functional linear regression model with functional predictor and scalar response. For this model, a procedure to test the slope function based on projecting the slope function onto an arbitrary L 2 basis has been introduced in the literature. We propose its bootstrap counterpart for testing the slope function, and obtain the asymptotic null distributions of the tests statistics and the asymptotic powers of the tests. Finally, we conduct a simulation study to evaluate the accuracy of the two tests procedures. As a practical illustration, we use the Export Development Bank of Iran dataset, and test the nullity of the slope function of a model predicting total annual noncurrent balance of facilities based on current balance of facilities. [ABSTRACT FROM AUTHOR]

We develop a group of robust, nonparametric hypothesis tests that detect differences between the covariance operators of several populations of functional data. These tests, called functional Kruskal–Wallis tests for covariance, or FKWC tests, are based on functional data depth ranks. FKWC tests work well even when the data are heavy‐tailed, which is shown both in simulation and theory. FKWC tests offer several other benefits: they have a simple asymptotic distribution under the null hypothesis, they are computationally cheap, and they possess transformation‐invariance properties. We show that under general alternative hypotheses, these tests are consistent under mild, nonparametric assumptions. As a result, we introduce a new functional depth function called L2‐root depth that works well for the purposes of detecting differences in magnitude between covariance kernels. We present an analysis of the FKWC test based on L2‐root depth under local alternatives. Through simulations, when the true covariance kernels have an infinite number of positive eigenvalues, we show that these tests have higher power than their competitors while maintaining their nominal size. We also provide a method for computing sample size and performing multiple comparisons. [ABSTRACT FROM AUTHOR]

In this paper we explore the basic properties of sub-Gaussian random variables and random elements. We also present various notions of subgaussianity (weak, - and -subgaussianity) of random elements with values in general Banach spaces. It is shown that the covariance operator of -subgaussian random element is Gaussian and some consequences of this result in spaces possessing certain geometric properties are noted. Moreover, the almost sure (a.s.) unconditional convergence of random series are considered and a sufficient condition of a.s. unconditional convergence of a random series of a special type with values in a Banach space with some geometric properties is proved. By the a.s. unconditional convergence of random series we understand the convergence of all rearrangements of the series on the same set of probability 1. With some effort, we prove one of the main results of the paper, which gives us a necessary condition for the a.s. unconditional convergence of random series of a special type in a general Banach space. For the proof, a lemma is used that establishes a connection between the moments of a random variable and which may be of independent interest. [ABSTRACT FROM AUTHOR]

In this paper we consider a Banach space valued random coefficient autoregressive process. Our studies on this process involve existence, weak law of large numbers, strong law of large numbers, complete convergence and central limit theorem. We follow the approach which is based on a suitable martingale coboundary decomposition in Banach space. [ABSTRACT FROM AUTHOR]

The envelope model is a recently developed methodology for multivariate analysis that enhances estimation accuracy by exploiting the relation between the mean and eigenstructure of the covariance matrix. We extend the envelope model to function-on-function linear regression, where the response and the predictor are assumed to be random functions in Hilbert spaces. We use a double envelope structure to accommodate the eigenstructures of the covariance operators for both the predictor and the response. The central idea is to establish a one-to-one relation between the functional envelope model and the multivariate envelope model and estimate the latter using an existing method. We also developed the asymptotic theories, confidence and prediction bands, an order determination method along with its consistency, and a characterization of the efficiency gain by the proposed model. Simulation comparisons with the standard function-on-function regression and data applications show significant improvement by our method in terms of cross-validated prediction error. for this article are available online. [ABSTRACT FROM AUTHOR]

We present a new approach known as greedy segmentation (GS) to identify multiple changepoints for a functional data sequence. The proposed multiple changepoint detection criterion links detectability with the projection onto a suitably chosen subspace and the changepoint locations. The changepoint estimator identifies the true changepoints for any predetermined number of changepoint candidates, either over-reporting or under-reporting. This theoretical finding supports the proposed GS estimator, which can be efficiently obtained in a greedy manner. The GS estimator's consistency holds without being restricted to the conventional at most one changepoint condition, and it is robust to the relative positions of the changepoints. Based on the GS estimator, the test statistic's asymptotic distribution leads to the novel GS algorithm, which identifies the number and locations of changepoints. Using intensive simulation studies, we compare the finite sample performance of the GS approach with other competing methods. We also apply our method to temporal changepoint detection in weather datasets. [ABSTRACT FROM AUTHOR]

The nonparametric estimation of covariance lies at the heart of functional data analysis, whether for curve or surface-valued data. The case of a two-dimensional domain poses both statistical and computational challenges, which are typically alleviated by assuming separability. However, separability is often questionable, sometimes even demonstrably inadequate. We propose a framework for the analysis of covariance operators of random surfaces that generalizes separability while retaining its major advantages. Our approach is based on the expansion of the covariance into a series of separable terms. The expansion is valid for any covariance over a two-dimensional domain. Leveraging the key notion of the partial inner product, we generalize the power iteration method to general Hilbert spaces, and show how the aforementioned expansion can be efficiently constructed in practice at the level of the surface observations. Truncation of the expansion and retention of the leading terms automatically induces a nonparametric estimator of the covariance, whose parsimony is dictated by the truncation level. The resulting estimator can be calculated, stored and manipulated with little computational overhead relative to separability. Consistency and rates of convergence are derived under mild regularity assumptions, illustrating the trade-off between bias and variance regulated by the truncation level. The merits and practical performance of the proposed methodology are demonstrated in a comprehensive simulation study. [ABSTRACT FROM AUTHOR]

In this article, we propose a general nonlinear sufficient dimension reduction (SDR) framework when both the predictor and the response lie in some general metric spaces. We construct reproducing kernel Hilbert spaces with kernels that are fully determined by the distance functions of the metric spaces, and then leverage the inherent structures of these spaces to define a nonlinear SDR framework. We adapt the classical sliced inverse regression within this framework for the metric space data. Next we build an estimator based on the corresponding linear operators, and show that it recovers the regression information in an unbiased manner. We derive the estimator at both the operator level and under a coordinate system, and establish its convergence rate. Lastly, we illustrate the proposed method using synthetic and real data sets that exhibit non-Euclidean geometry. [ABSTRACT FROM AUTHOR]

We prove an analogue of Khinchin's theorem for weakly correlated random elements with values in the spaces lp, 1 ⩽ p < ∞. [ABSTRACT FROM AUTHOR]

We introduce a statistical graphical model for multivariate functional data, which are common in medical applications such as EEG and fMRI. Recently published functional graphical models rely on the multivariate Gaussian process assumption, but we relax it by introducing the functional copula Gaussian graphical model (FCGGM). This model removes the marginal Gaussian assumption but retains the simplicity of the Gaussian dependence structure, which is particularly attractive for large data. We develop four estimators for the FCGGM and establish the consistency and the convergence rates of one of them. We compare our FCGGM with the existing functional Gaussian graphical model by simulations, and apply our method to an EEG dataset to construct brain networks. for this article are available online. [ABSTRACT FROM AUTHOR]

In this paper we propose statistical inference tools for the covariance operators of functional time series in the two sample and change point problem. In contrast to most of the literature, the focus of our approach is not testing the null hypothesis of exact equality of the covariance operators. Instead, we propose to formulate the null hypotheses in the form that "the distance between the operators is small", where we measure deviations by the sup-norm. We provide powerful bootstrap tests for these type of hypotheses, investigate their asymptotic properties and study their finite sample properties by means of a simulation study. [ABSTRACT FROM AUTHOR]

Kernel methods provide an elegant framework for developing nonlinear learning algorithms from simple linear methods. Though these methods have superior empirical performance in several real data applications, their usefulness is inhibited by the significant computational burden incurred in large sample situations. Various approximation schemes have been proposed in the literature to alleviate these computational issues, and the approximate kernel machines are shown to retain the empirical performance. However, the theoretical properties of these approximate kernel machines are less well understood. In this work, we theoretically study the trade-off between computational complexity and statistical accuracy in Nyström approximate kernel principal component analysis (KPCA), wherein we show that the Nyström approximate KPCA matches the statistical performance of (nonapproximate) KPCA while remaining computationally beneficial. Additionally, we show that Nyström approximate KPCA outperforms the statistical behavior of another popular approximation scheme, the random feature approximation, when applied to KPCA. [ABSTRACT FROM AUTHOR]

The law of large numbers for weakly correlated random elements with values in a separable Hilbert space is proved. [ABSTRACT FROM AUTHOR]

Functional principal component analysis for sparse longitudinal data usually proceeds by first smoothing the covariance surface, and then obtaining an eigendecomposition of the associated covariance operator. Here we consider the use of penalized tensor product splines for the initial smoothing step. Drawing on a result regarding finite-rank symmetric integral operators, we derive an explicit spline representation of the estimated eigenfunctions, and show that the effect of penalization can be notably disparate for alternative approaches to tensor product smoothing. The latter phenomenon is illustrated with two data sets derived from magnetic resonance imaging of the human brain. • Tensor product splines are often used for functional principal component analysis. • We examine the shrinkage behavior of two alternative penalization schemes. • We illustrate how choice of penalization method can markedly impact the results. [ABSTRACT FROM AUTHOR]

For testing hypothesis on the covariance operator of functional time series, we suggest to use the full functional information and to avoid dimension reduction techniques. The limit distribution follows from the central limit theorem of the weak convergence of the partial sum process in general Hilbert space applied to the product space. In order to obtain critical values for tests, we generalise bootstrap results from the independent to the dependent case. This results can be applied to covariance operators, autocovariance operators and cross covariance operators. We discuss one sample and changepoint tests and give some simulation results. [ABSTRACT FROM AUTHOR]

Functional canonical correlation analysis (FCCA) has been applied in many contexts, but the asymptotic properties have not yet been studied enough. In this paper we consider a general setup resembling that of Eubank and Hsing (2008). We focus on the convergence of estimates of the associated canonical functions to their population counterparts. A regularized estimator is proposed based on the Tikhinov regularized approach. Under some conditions, an upper bound is derived for this estimator and furthermore, a sharp lower bound is established. Consequently, the minimax optimal rate is obtained and depends on the level of dependence between the two stochastic processes. A simple simulation is performed to illustrate our methods. • We study functional canonical correlation analysis in a more general framework. • We establish the minimax rate for the canonical functions. • We conduct numerical simulation to illustrate the performance of the estimator. [ABSTRACT FROM AUTHOR]

In this paper, we present a mathematical and computational framework for comparing and matching distributions in reproducing kernel Hilbert spaces (RKHS). This framework, called optimal transport in RKHS, is a generalization of the optimal transport problem in input spaces to (potentially) infinite-dimensional feature spaces. We provide a computable formulation of Kantorovich's optimal transport in RKHS. In particular, we explore the case in which data distributions in RKHS are Gaussian, obtaining closed-form expressions of both the estimated Wasserstein distance and optimal transport map via kernel matrices. Based on these expressions, we generalize the Bures metric on covariance matrices to infinite-dimensional settings, providing a new metric between covariance operators. Moreover, we extend the correlation alignment problem to Hilbert spaces, giving a new strategy for matching distributions in RKHS. Empirically, we apply the derived formulas under the Gaussianity assumption to image classification and domain adaptation. In both tasks, our algorithms yield state-of-the-art performances, demonstrating the effectiveness and potential of our framework. [ABSTRACT FROM AUTHOR]

A basic problem in operator theory is to estimate how a small perturbation affects the eigenspaces of a self-adjoint compact operator. In this paper, we prove upper bounds for the subspace distance, tailored for relative perturbations. As a main example, we consider the empirical covariance operator and show that a sharp bound can be achieved under a relative gap condition. The proof is based on a novel contraction phenomenon, contrasting previous spectral perturbation approaches. [ABSTRACT FROM AUTHOR]

Covariance operators are fundamental concepts and modelling tools for many functional data analysis methods, such as functional principal component analysis. However, the empirical (or estimated) covariance operator becomes too costly to compute when the functional dataset gets big. This paper studies a randomized algorithm for covariance operator estimation. The algorithm works by sampling and rescaling observations from the large functional data collection to form a sketch of much smaller size and performs computation on the sketch to obtain the subsampled empirical covariance operator. The proposed algorithm is theoretically justified via nonasymptotic bounds between the subsampled and the full‐sample empirical covariance operator in terms of the Hilbert‐Schmidt norm and the operator norm. It is shown that the optimal sampling probability that minimizes the expected squared Hilbert‐Schmidt norm of the subsampling error is determined by the norm of each function. Simulated and real data examples are used to illustrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we provide sufficient conditions ensuring that a ѱ-mixing property holds for the sequence of empirical cumulative distribution functions associated with a conjugate process. Numerical examples are also provided to illustrate our results. [ABSTRACT FROM AUTHOR]

We consider nonparametric estimation of a covariance function on the unit square, given a sample of discretely observed fragments of functional data. When each sample path is observed only on a subinterval of length |$\delta<1$|⁠, one has no statistical information on the unknown covariance outside a |$\delta$| -band around the diagonal. The problem seems unidentifiable without parametric assumptions, but we show that nonparametric estimation is feasible under suitable smoothness and rank conditions on the unknown covariance. This remains true even when the observations are discrete, and we give precise deterministic conditions on how fine the observation grid needs to be relative to the rank and fragment length for identifiability to hold true. We show that our conditions translate the estimation problem to a low-rank matrix completion problem, construct a nonparametric estimator in this vein, and study its asymptotic properties. We illustrate the numerical performance of our method on real and simulated data. [ABSTRACT FROM AUTHOR]

Abstract The covariance operator plays an important role in modern statistical methods and is critical for inference. It is most often estimated by the empirical covariance operator. In spite of its simple and appealing properties, however, this estimator can be improved by a class of shrinkage operators. In this paper, we study shrinkage estimation of the covariance operator in reproducing kernel Hilbert spaces. A data-driven shrinkage estimator enjoying desirable theoretical and computational properties is proposed. The procedure is easily implemented and its numerical performance is investigated through simulations. In finite samples, the estimator outperforms the empirical covariance operator, especially when the data dimension is much larger than the sample size. We also show that the rate of convergence in Hilbert–Schmidt norm is of the order n − 1 ∕ 2. Furthermore, we establish the minimax optimal rate of convergence over suitable classes of probability measures and demonstrate that these shrinkage operators are all minimax rate-optimal. [ABSTRACT FROM AUTHOR]

Is it possible to detect if the sample paths of a stochastic process almost surely admit a finite expansion with respect to some/any basis? The determination is to be made on the basis of a finite collection of discretely/noisily observed sample paths. We show that it is indeed possible to construct a hypothesis testing scheme that is almost surely guaranteed to make only finite many incorrect decisions as more data are collected. Said differently, our scheme almost certainly detects whether the process has a finite or infinite basis expansion for all sufficiently large sample sizes. Our approach relies on Cover's classical test for the irrationality of a mean, combined with tools for the non-parametric estimation of covariance operators. [ABSTRACT FROM AUTHOR]