Search

Goodness-of-fit testing is often criticized for its lack of practical relevance; since ``all models are wrong'', the null hypothesis that the data conform to our model is ultimately always rejected when the sample size is large enough. Despite this, probabilistic models are still used extensively, raising the more pertinent question of whether the model is good enough for a specific task. This question can be formalized as a robust goodness-of-fit testing problem by asking whether the data were generated by a distribution corresponding to our model up to some mild perturbation. In this paper, we show that existing kernel goodness-of-fit tests are not robust according to common notions of robustness including qualitative and quantitative robustness. We also show that robust techniques based on tilted kernels from the parameter estimation literature are not sufficient for ensuring both types of robustness in the context of goodness-of-fit testing. We therefore propose the first robust kernel goodness-of-fit test which resolves this open problem using kernel Stein discrepancy balls, which encompass perturbation models such as Huber contamination models and density uncertainty bands., Comment: 51 pages, 14 figures

In the statistical literature, as well as in artificial intelligence and machine learning, measures of discrepancy between two probability distributions are largely used to develop measures of goodness-of-fit. We concentrate on quadratic distances, which depend on a non-negative definite kernel. We propose a unified framework for the study of two-sample and k-sample goodness of fit tests based on the concept of matrix distance. We provide a succinct review of the goodness of fit literature related to the use of distance measures, and specifically to quadratic distances. We show that the quadratic distance kernel-based two-sample test has the same functional form with the maximum mean discrepancy test. We develop tests for the $k$-sample scenario, where the two-sample problem is a special case. We derive their asymptotic distribution under the null hypothesis and discuss computational aspects of the test procedures. We assess their performance, in terms of level and power, via extensive simulations and a real data example. The proposed framework is implemented in the QuadratiK package, available in both R and Python environments., Comment: 35 pages, 18 figures

The paper presents a new statistical method that enables the use of systematic errors in the maximum-likelihood regression of integer-count Poisson data to a parametric model. The method is primarily aimed at the characterization of the goodness-of-fit statistic in the presence of the over-dispersion that is induced by sources of systematic error, and is based on a quasi-maximum-likelihood method that retains the Poisson distribution of the data. We show that the Poisson deviance, which is the usual goodness-of-fit statistic and that is commonly referred to in astronomy as the Cash statistics, can be easily generalized in the presence of systematic errors, under rather general conditions. The method and the associated statistics are first developed theoretically, and then they are tested with the aid of numerical simulations and further illustrated with real-life data from astronomical observations. The statistical methods presented in this paper are intended as a simple general-purpose framework to include additional sources of uncertainty for the analysis of integer-count data in a variety of practical data analysis situations., Comment: ApJ accepted

Despite the wide usage of parametric point processes in theory and applications, a sound goodness-of-fit procedure to test whether a given parametric model is appropriate for data coming from a self-exciting point processes has been missing in the literature. In this work, we establish a bootstrap-based goodness-of-fit test which empirically works for all kinds of self-exciting point processes (and even beyond). In an infill-asymptotic setting we also prove its asymptotic consistency, albeit only in the particular case that the underlying point process is inhomogeneous Poisson.

A type of dynamic network involves temporally ordered interactions between actors, where past network configurations may influence future ones. The relational event model can be used to identify the underlying dynamics that drive interactions among system components. Despite the rapid development of this model over the past 15 years, an ongoing area of research revolves around evaluating the goodness of fit of this model, especially when it incorporates time-varying and random effects. Current methodologies often rely on comparing observed and simulated events using specific statistics, but this can be computationally intensive, and requires various assumptions. We propose an additive mixed-effect relational event model estimated via case-control sampling, and introduce a versatile framework for testing the goodness of fit of such models using weighted martingale residuals. Our focus is on a Kolmogorov-Smirnov type test designed to assess if covariates are accurately modeled. Our approach can be easily extended to evaluate whether other features of network dynamics have been appropriately incorporated into the model. We assess the goodness of fit of various relational event models using synthetic data to evaluate the test's power and coverage. Furthermore, we apply the method to a social study involving 57,791 emails sent by 159 employees of a Polish manufacturing company in 2010. The method is implemented in the R package mgcv.

Two new goodness of fit tests for the Pareto type-I distribution for complete and right censored data are proposed using fixed point characterization based on Steins type identity. The asymptotic distributions of the test statistics under both the null and alternative hypotheses are obtained. The performance of the proposed tests is evaluated and compared with existing tests through a Monte Carlo simulation experiment. The newly proposed tests exhibit greater power than existing tests for the Pareto type-I distribution. Finally, the methodology is applied to real-world data sets., Comment: 33 Pages

This paper establishes the functional convergence of the Extreme Nelson--Aalen and Extreme Kaplan--Meier estimators, which are designed to capture the heavy-tailed behaviour of censored losses. The resulting limit representations can be used to obtain the distributions of pathwise functionals with respect to the so-called tail process. For instance, we may recover the convergence of a censored Hill estimator, and we further investigate two goodness-of-fit statistics for the tail of the loss distribution. Using the the latter limit theorems, we propose two rules for selecting a suitable number of order statistics, both based on test statistics derived from the functional convergence results. The effectiveness of these selection rules is investigated through simulations and an application to a real dataset comprised of French motor insurance claim sizes.

The cumulative residual extropy (CRJ) is a measure of uncertainty that serves as an alternative to extropy. It replaces the probability density function with the survival function in the expression of extropy. This work introduces a new concept called normalized dynamic survival extropy (NDSE), a dynamic variation of CRJ. We observe that NDSE is equivalent to CRJ of the random variable of interest $X_{[t]}$ in the age replacement model at a fixed time $t$. Additionally, we have demonstrated that NDSE remains constant exclusively for exponential distribution at any time. We categorize two classes, INDSE and DNDSE, based on their increasing and decreasing NDSE values. Next, we present a non-parametric test to assess whether a distribution follows an exponential pattern against INDSE. We derive the exact and asymptotic distribution for the test statistic $\widehat{\Delta}^*$. Additionally, a test for asymptotic behavior is presented in the paper for right censoring data. Finally, we determine the critical values and power of our exact test through simulation. The simulation demonstrates that the suggested test is easy to compute and has significant statistical power, even with small sample sizes. We also conduct a power comparison analysis among other tests, which shows better power for the proposed test against other alternatives mentioned in this paper. Some numerical real-life examples validating the test are also included.

A wide range of (multivariate) temporal (1D) and spatial (2D) data analysis tasks, such as grouping vehicle sensor trajectories, can be formulated as clustering with given metric constraints. Existing metric-constrained clustering algorithms overlook the rich correlation between feature similarity and metric distance, i.e., metric autocorrelation. The model-based variations of these clustering algorithms (e.g. TICC and STICC) achieve SOTA performance, yet suffer from computational instability and complexity by using a metric-constrained Expectation-Maximization procedure. In order to address these two problems, we propose a novel clustering algorithm, MC-GTA (Model-based Clustering via Goodness-of-fit Tests with Autocorrelations). Its objective is only composed of pairwise weighted sums of feature similarity terms (square Wasserstein-2 distance) and metric autocorrelation terms (a novel multivariate generalization of classic semivariogram). We show that MC-GTA is effectively minimizing the total hinge loss for intra-cluster observation pairs not passing goodness-of-fit tests, i.e., statistically not originating from the same distribution. Experiments on 1D/2D synthetic and real-world datasets demonstrate that MC-GTA successfully incorporates metric autocorrelation. It outperforms strong baselines by large margins (up to 14.3% in ARI and 32.1% in NMI) with faster and stabler optimization (>10x speedup)., Comment: ICML-2024 Proceedings

Many flexible families of positive random variables exhibit non-closed forms of the density and distribution functions and this feature is considered unappealing for modelling purposes. However, such families are often characterized by a simple expression of the corresponding Laplace transform. Relying on the Laplace transform, we propose to carry out parameter estimation and goodness-of-fit testing for a general class of non-standard laws. We suggest a novel data-driven inferential technique, providing parameter estimators and goodness-of-fit tests, whose large-sample properties are derived. The implementation of the method is specifically considered for the positive stable and Tweedie distributions. A Monte Carlo study shows good finite-sample performance of the proposed technique for such laws.

A goodness-of-fit index measures the consistency of consumption data with a given model of utility-maximization. We show that for the class of well-behaved (i.e., continuous and increasing) utility functions there is no goodness-of-fit index that is continuous and accurate, where the latter means that a perfect score is obtained if and only if a dataset can be rationalized by a well-behaved utility function. While many standard goodness-of-fit indices are inaccurate we show that these indices are (in a sense we make precise) essentially accurate. Goodness-of-fit indices are typically generated by loss functions and we find that standard loss functions usually do not yield a best-fitting utility function when they are minimized. Nonetheless, welfare comparisons can be made by working out a robust preference relation from the data.

We explore the minimax optimality of goodness-of-fit tests on general domains using the kernelized Stein discrepancy (KSD). The KSD framework offers a flexible approach for goodness-of-fit testing, avoiding strong distributional assumptions, accommodating diverse data structures beyond Euclidean spaces, and relying only on partial knowledge of the reference distribution, while maintaining computational efficiency. We establish a general framework and an operator-theoretic representation of the KSD, encompassing many existing KSD tests in the literature, which vary depending on the domain. We reveal the characteristics and limitations of KSD and demonstrate its non-optimality under a certain alternative space, defined over general domains when considering $\chi^2$-divergence as the separation metric. To address this issue of non-optimality, we propose a modified, minimax optimal test by incorporating a spectral regularizer, thereby overcoming the shortcomings of standard KSD tests. Our results are established under a weak moment condition on the Stein kernel, which relaxes the bounded kernel assumption required by prior work in the analysis of kernel-based hypothesis testing. Additionally, we introduce an adaptive test capable of achieving minimax optimality up to a logarithmic factor by adapting to unknown parameters. Through numerical experiments, we illustrate the superior performance of our proposed tests across various domains compared to their unregularized counterparts., Comment: 54 pages

In this paper we present the methodology for detecting outliers and testing the goodness-of-fit of random sets using topological data analysis. We construct the filtration from level sets of the signed distance function and consider various summary functions of the persistence diagram derived from the obtained persistence homology. The outliers are detected using functional depths for the summary functions. Global envelope tests using the summary statistics as test statistics were used to construct the goodness-of-fit test. The procedures were justified by a simulation study using germ-grain random set models., Comment: 36 pages, 12 figures, 21 tables

Thousands of experiments are analyzed and papers are published each year involving the statistical analysis of grouped data. While this area of statistics is often perceived - somewhat naively - as saturated, several misconceptions still affect everyday practice, and new frontiers have so far remained unexplored. Researchers must be aware of the limitations affecting their analyses and what are the new possibilities in their hands. Motivated by this need, the article introduces a unifying approach to the analysis of grouped data which allows us to study the class of divisible statistics - that includes Pearson's $\chi^2$, the likelihood ratio as special cases - with a fresh perspective. The contributions collected in this manuscript span from modeling and estimation to distribution-free goodness-of-fit tests. Perhaps the most surprising result presented here is that, in a sparse regime, all tests proposed in the literature are dominated by a class of weighted linear statistics.

Financial firms often rely on factor models to explain correlations among asset returns. These models are important for managing risk, for example by modeling the probability that many assets will simultaneously lose value. Yet after major events, e.g., COVID-19, analysts may reassess whether existing models continue to fit well: specifically, after accounting for the factor exposures, are the residuals of the asset returns independent? With this motivation, we introduce the mosaic permutation test, a nonparametric goodness-of-fit test for preexisting factor models. Our method allows analysts to use nearly any machine learning technique to detect model violations while provably controlling the false positive rate, i.e., the probability of rejecting a well-fitting model. Notably, this result does not rely on asymptotic approximations and makes no parametric assumptions. This property helps prevent analysts from unnecessarily rebuilding accurate models, which can waste resources and increase risk. We illustrate our methodology by applying it to the Blackrock Fundamental Equity Risk (BFRE) model. Using the mosaic permutation test, we find that the BFRE model generally explains the most significant correlations among assets. However, we find evidence of unexplained correlations among certain real estate stocks, and we show that adding new factors improves model fit. We implement our methods in the python package mosaicperm., Comment: 38 pages, 13 figures

We present a novel data-oriented statistical framework that assesses the presumed Gaussian dependence structure in a pairwise setting. This refers to both multivariate normality and normal copula goodness-of-fit testing. The proposed test clusters the data according to the 20/60/20 rule and confronts conditional covariance (or correlation) estimates on the obtained subsets. The corresponding test statistic has a natural practical interpretation, desirable statistical properties, and asymptotic pivotal distribution under the multivariate normality assumption. We illustrate the usefulness of the introduced framework using extensive power simulation studies and show that our approach outperforms popular benchmark alternatives. Also, we apply the proposed methodology to commodities market data.

Based on $m$-fold integrated empirical measures, we study three new classes of goodness-of-fits tests, generalizing Anderson-Darling, Cram\'er-von Mises, and Watson statistics, respectively, and examine the corresponding limiting stochastic processes. The limiting null distributions of the statistics all lead to explicitly solvable cases with closed-form expressions for the corresponding Karhunen-Lo\`{e}ve expansions and covariance kernels. In particular, the eigenvalues are shown to be $\frac1{k(k+1)\cdots (k+2m-1)}$ for the generalized Anderson-Darling, $\frac1{(\pi k)^{2m}}$ for the generalized Cram\'er-von Mises, and $\frac1{2\pi\lceil k/2\rceil^{2m}}$ for the generalized Watson statistics, respectively. The infinite products of the resulting moment generating functions are further simplified to finite ones so as to facilitate efficient numerical calculations. These statistics are capable of detecting different features of the distributions and thus provide a useful toolbox for goodness-of-fit testing., Comment: 71 pages, 10 figures

Traditional methods of model diagnostics may include a plethora of graphical techniques based on residual analysis, as well as formal tests (e.g. Shapiro-Wilk test for normality and Bartlett test for homogeneity of variance). In this paper we derive a new distance metric based on the half-normal plot with a simulation envelope, a graphical model evaluation method, and investigate its properties through simulation studies. The proposed metric can help to assess the fit of a given model, and also act as a model selection criterion by being comparable across models, whether based or not on a true likelihood. More specifically, it quantitatively encompasses the model evaluation principles and removes the subjective bias when closely related models are involved. We validate the technique by means of an extensive simulation study carried out using count data, and illustrate with two case studies in ecology and fisheries research.

A fundamental and often final step in time series modeling is to assess the quality of fit of a proposed model to the data. Since the underlying distribution of the innovations that generate a model is often not prescribed, goodness-of-fit tests typically take the form of testing the fitted residuals for serial independence. However, these fitted residuals are inherently dependent since they are based on the same parameter estimates and thus standard tests of serial independence, such as those based on the autocorrelation function (ACF) or distance correlation function (ADCF) of the fitted residuals need to be adjusted. The sample splitting procedure in Pfister et al.~(2018) is one such fix for the case of models for independent data, but fails to work in the dependent setting. In this paper sample splitting is leveraged in the time series setting to perform tests of serial dependence of fitted residuals using the ACF and ADCF. Here the first $f_n$ of the data points are used to estimate the parameters of the model and then using these parameter estimates, the last $l_n$ of the data points are used to compute the estimated residuals. Tests for serial independence are then based on these $l_n$ residuals. As long as the overlap between the $f_n$ and $l_n$ data splits is asymptotically 1/2, the ACF and ADCF tests of serial independence tests often have the same limit distributions as though the underlying residuals are indeed iid. In particular if the first half of the data is used to estimate the parameters and the estimated residuals are computed for the entire data set based on these parameter estimates, then the ACF and ADCF can have the same limit distributions as though the residuals were iid. This procedure ameliorates the need for adjustment in the construction of confidence bounds for both the ACF and ADCF in goodness-of-fit testing., Comment: 31 pages, 4 figures, 1 table

We introduce the QuadratiK package that incorporates innovative data analysis methodologies. The presented software, implemented in both R and Python, offers a comprehensive set of goodness-of-fit tests and clustering techniques using kernel-based quadratic distances, thereby bridging the gap between the statistical and machine learning literatures. Our software implements one, two and k-sample tests for goodness of fit, providing an efficient and mathematically sound way to assess the fit of probability distributions. Expanded capabilities of our software include supporting tests for uniformity on the d-dimensional Sphere based on Poisson kernel densities. Particularly noteworthy is the incorporation of a unique clustering algorithm specifically tailored for spherical data that leverages a mixture of Poisson kernel-based densities on the sphere. Alongside this, our software includes additional graphical functions, aiding the users in validating, as well as visualizing and representing clustering results. This enhances interpretability and usability of the analysis. In summary, our R and Python packages serve as a powerful suite of tools, offering researchers and practitioners the means to delve deeper into their data, draw robust inference, and conduct potentially impactful analyses and inference across a wide array of disciplines., Comment: 36 pages, 9 figures

This paper introduces a novel goodness-of-fit test technique for parametric conditional distributions. The proposed tests are based on a residual marked empirical process, for which we develop a conditional Principal Component Analysis. The obtained components provide a basis for various types of new tests in addition to the omnibus one. Component tests that based on each component serve as experts in detecting certain directions. Smooth tests that assemble a few components are also of great use in practice. To further improve testing efficiency, we introduce a component selection approach, aiming to identify the most contributory components. The finite sample performance of the proposed tests is illustrated through Monte Carlo experiments.

Among the various models designed for dependent count data, integer-valued autoregressive (INAR) processes enjoy great popularity. Typically, statistical inference for INAR models uses asymptotic theory that relies on rather stringent (parametric) assumptions on the innovations such as Poisson or negative binomial distributions. In this paper, we present a novel semi-parametric goodness-of-fit test tailored for the INAR model class. Relying on the INAR-specific shape of the joint probability generating function, our approach allows for model validation of INAR models without specifying the (family of the) innovation distribution. We derive the limiting null distribution of our proposed test statistic, prove consistency under fixed alternatives and discuss its asymptotic behavior under local alternatives. By manifold Monte Carlo simulations, we illustrate the overall good performance of our testing procedure in terms of power and size properties. In particular, it turns out that the power can be considerably improved by using higher-order test statistics. We conclude the article with the application on three real-world economic data sets., Comment: 38 pages, 6 figures, 11 tables

The problem of assessing a parametric regression model in the presence of spatial correlation is addressed in this work. For that purpose, a goodness-of-fit test based on a $L_2$-distance comparing a parametric and a nonparametric regression estimators is proposed. Asymptotic properties of the test statistic, both under the null hypothesis and under local alternatives, are derived. Additionally, a bootstrap procedure is designed to calibrate the test in practice. Finite sample performance of the test is analyzed through a simulation study, and its applicability is illustrated using a real data example., Comment: 49 pages, 7 figures