Search

Existing approaches to model uncertainty typically either compare models using a quantitative model selection criterion or evaluate posterior model probabilities having set a prior. In this paper, we propose an alternative strategy which views missing observations as the source of model uncertainty, where the true model would be identified with the complete data. To quantify model uncertainty, it is then necessary to provide a probability distribution for the missing observations conditional on what has been observed. This can be set sequentially using one-step-ahead predictive densities, which recursively sample from the best model according to some consistent model selection criterion. Repeated predictive sampling of the missing data, to give a complete dataset and hence a best model each time, provides our measure of model uncertainty. This approach bypasses the need for subjective prior specification or integration over parameter spaces, addressing issues with standard methods such as the Bayes factor. Predictive resampling also suggests an alternative view of hypothesis testing as a decision problem based on a population statistic, where we directly index the probabilities of competing models. In addition to hypothesis testing, we provide illustrations from density estimation and variable selection, demonstrating our approach on a range of standard problems., Comment: 44 pages, 25 figures, 1 table

The family of log-concave density functions contains various kinds of common probability distributions. Due to the shape restriction, it is possible to find the nonparametric estimate of the density, for example, the nonparametric maximum likelihood estimate (NPMLE). However, the associated uncertainty quantification of the NPMLE is less well developed. The current techniques for uncertainty quantification are Bayesian, using a Dirichlet process prior combined with the use of Markov chain Monte Carlo (MCMC) to sample from the posterior. In this paper, we start with the NPMLE and use a version of the martingale posterior distribution to establish uncertainty about the NPMLE. The algorithm can be implemented in parallel and hence is fast. We prove the convergence of the algorithm by constructing suitable submartingales. We also illustrate results with different models and settings and some real data, and compare our method with that within the literature., Comment: 12 pages, 7 figures

This paper proposes a new nonparametric Bayesian bootstrap for a mixture model, by developing the traditional Bayesian bootstrap. We first reinterpret the Bayesian bootstrap, which uses the P\'olya-urn scheme, as a gradient ascent algorithm which associated one-step solver. The key then is to use the same basic mechanism as the Bayesian bootstrap with the switch from a point mass kernel to a continuous kernel. Just as the Bayesian bootstrap works solely from the empirical distribution function, so the new Bayesian bootstrap for mixture models works off the nonparametric maximum likelihood estimator for the mixing distribution. From a theoretical perspective, we prove the convergence and exchangeability of the sample sequences from the algorithm and also illustrate our results with different models and settings and some real data., Comment: 16 pages, 6 figures

The paper considers a Cox process where the stochastic intensity function for the Poisson data model is itself a non-homogeneous Poisson process. We show that it is possible to obtain the marginal data process, namely a non-homogeneous count process exhibiting over-dispersion. While the intensity function is non-decreasing, it is straightforward to transform the data so that a non-decreasing intensity function is appropriate. We focus on a time series for arrival times of a process and, in particular, we are able to find an exact form for the marginal probability for the observed data, so allowing for an easy to implement estimation algorithm via direct calculations of the likelihood function.

The construction of objective priors is, at best, challenging for multidimensional parameter spaces. A common practice is to assume independence and set up the joint prior as the product of marginal distributions obtained via "standard" objective methods, such as Jeffreys or reference priors. However, the assumption of independence a priori is not always reasonable, and whether it can be viewed as strictly objective is still open to discussion. In this paper, by extending a previously proposed objective approach based on scoring rules for the one dimensional case, we propose a novel objective prior for multidimensional parameter spaces which yields a dependence structure. The proposed prior has the appealing property of being proper and does not depend on the chosen model; only on the parameter space considered., Comment: 20 pages, 5 figures, 10 tables

Deterministic compartmental models are predominantly used in the modeling of infectious diseases, though stochastic models are considered more realistic, yet are complicated to estimate due to missing data. In this paper we present a novel algorithm for estimating the stochastic SIR/SEIR epidemic model within a Bayesian framework, which can be readily extended to more complex stochastic compartmental models. Specifically, based on the infinitesimal conditional independence properties of the model, we are able to find a proposal distribution for a Metropolis algorithm which is very close to the correct posterior distribution. As a consequence, rather than perform a Metropolis step updating one missing data point at a time, as in the current benchmark Markov chain Monte Carlo (MCMC) algorithm, we are able to extend our proposal to the entire set of missing observations. This improves the MCMC methods dramatically and makes the stochastic models now a viable modeling option. A number of real data illustrations and the necessary mathematical theory supporting our results are presented.

The paper presents a new perspective on the mixture of Dirichlet process model which allows the recovery of full and correct uncertainty quantification associated with the full model, even after having integrated out the random distribution function. The implication is that we can run a simple Markov chain Monte Carlo algorithm and subsequently return the original uncertainty which was removed from the integration. This also has the benefit of avoiding more complicated algorithms which do not perform the integration step. Numerous illustrations are presented.

We present simple conditions for Bayesian consistency in the supremum metric. The key to the technique is a triangle inequality which allows us to explicitly use weak convergence, a consequence of the standard Kullback--Leibler support condition for the prior. A further condition is to ensure that smoothed versions of densities are not too far from the original density, thus dealing with densities which could track the data too closely. A key result of the paper is that we demonstrate supremum consistency using weaker conditions compared to those currently used to secure $\mathbb{L}_1$ consistency., Comment: 11 pages

We introduce a class of integral theorems based on cyclic functions and Riemann sums approximating integrals. The Fourier integral theorem, derived as a combination of a transform and inverse transform, arises as a special case. The integral theorems provide natural estimators of density functions via Monte Carlo methods. Assessments of the quality of the density estimators can be used to obtain optimal cyclic functions, alternatives to the sin function, which minimize square integrals. Our proof techniques rely on a variational approach in ordinary differential equations and the Cauchy residue theorem in complex analysis., Comment: 21 pages, 5 figures. arXiv admin note: text overlap with arXiv:2106.06608

Taking the Fourier integral theorem as our starting point, in this paper we focus on natural Monte Carlo and fully nonparametric estimators of multivariate distributions and conditional distribution functions. We do this without the need for any estimated covariance matrix or dependence structure between variables. These aspects arise immediately from the integral theorem. Being able to model multivariate data sets using conditional distribution functions we can study a number of problems, such as prediction for Markov processes, estimation of mixing distribution functions which depend on covariates, and general multivariate data. Estimators are explicit Monte Carlo based and require no recursive or iterative algorithms., Comment: 20 pages, 10 figures

The paper presents a novel asymptotic distribution for a mle when the log--likelihood is strictly concave in the parameter for all data points; for example, the exponential family. The new asymptotic distribution can be seen as a refinement of the usual normal asymptotic distribution and is comparable to an Edgeworth expansion. However, it is obtained with weaker conditions than even those for asymptotic normality. The same technique is then used to find the exact distribution of the weighted likelihood bootstrap sampler.

In this paper we present a new measure for the overlap of two density functions which provides motivation and interpretation currently lacking with benchmark measures based on the proportion of similar response, also known as the overlap coefficient. We use this new measure to present an alternative to the $p$--value as a guide to the choice of treatment in a comparative trial; where a current treatment and a new treatment are undergoing investigation. We show that it is possible to reject the null hypothesis; i.e. the new treatment is significantly different in response to the old treatment, while the proposed new summary for the same experiment indicates that as low as one in ten individuals subject to the new treatment behave differently to individuals on the old one.

In this paper we introduce a novel objective prior distribution levering on the connections between information, divergence and scoring rules. In particular, we do so from the starting point of convex functions representing information in density functions. This provides a natural route to proper local scoring rules using Bregman divergence. Specifically, we determine the prior which solves setting the score function to be a constant. While in itself this provides motivation for an objective prior, the prior also minimizes a corresponding information criterion.

The prior distribution on parameters of a sampling distribution is the usual starting point for Bayesian uncertainty quantification. In this paper, we present a different perspective which focuses on missing observations as the source of statistical uncertainty, with the parameter of interest being known precisely given the entire population. We argue that the foundation of Bayesian inference is to assign a distribution on missing observations conditional on what has been observed. In the conditionally i.i.d. setting with an observed sample of size $n$, the Bayesian would thus assign a predictive distribution on the missing $Y_{n+1:\infty}$ conditional on $Y_{1:n}$, which then induces a distribution on the parameter. Demonstrating an application of martingales, Doob shows that choosing the Bayesian predictive distribution returns the conventional posterior as the distribution of the parameter. Taking this as our cue, we relax the predictive machine, avoiding the need for the predictive to be derived solely from the usual prior to posterior to predictive density formula. We introduce the \textit{martingale posterior distribution}, which returns Bayesian uncertainty directly on any statistic of interest without the need for the likelihood and prior, and this distribution can be sampled through a computational scheme we name \textit{predictive resampling}. To that end, we introduce new predictive methodologies for multivariate density estimation, regression and classification that build upon recent work on bivariate copulas., Comment: 62 pages, 22 figures, 3 tables; added discussion on frequentist consistency, included more context, added references

With the vast amount of data collected on football and the growth of computing abilities, many games involving decision choices can be optimized. The underlying rule is the maximization of an expected utility of outcomes and the law of large numbers. The data available allows us to compute with high accuracy the probabilities of outcomes of decisions and the well defined points system in the game allows us to have the necessary terminal utilities. With some well established theory we can then optimize choices at a single play level., Comment: 62 pages, 25 figures

Starting with the Fourier integral theorem, we present natural Monte Carlo estimators of multivariate functions including densities, mixing densities, transition densities, regression functions, and the search for modes of multivariate density functions (modal regression). Rates of convergence are established and, in many cases, provide superior rates to current standard estimators such as those based on kernels, including kernel density estimators and kernel regression functions. Numerical illustrations are presented., Comment: 58 pages, 6 figures

In this paper we introduce a new sampling algorithm which has the potential to be adopted as a universal replacement to the Metropolis--Hastings algorithm. It is related to the slice sampler, and motivated by an algorithm which is applicable to discrete probability distributions %which can be viewed as an alternative to the Metropolis--Hastings algorithm in this setting, which obviates the need for a proposal distribution, in that is has no accept/reject component. This paper looks at the continuous counterpart. A latent variable combined with a slice sampler and a shrinkage procedure applied to uniform density functions creates a highly efficient sampler which can generate random variables from very high dimensional distributions as a single block., Comment: 19 pages, 10 figures

In this paper, we use the class of Wasserstein metrics to study asymptotic properties of posterior distributions. Our first goal is to provide sufficient conditions for posterior consistency. In addition to the well-known Schwartz's Kullback--Leibler condition on the prior, the true distribution and most probability measures in the support of the prior are required to possess moments up to an order which is determined by the order of the Wasserstein metric. We further investigate convergence rates of the posterior distributions for which we need stronger moment conditions. The required tail conditions are sharp in the sense that the posterior distribution may be inconsistent or contract slowly to the true distribution without these conditions. Our study involves techniques that build on recent advances on Wasserstein convergence of empirical measures. We apply the results to density estimation with a Dirichlet process mixture prior and conduct a simulation study for further illustration., Comment: 43pages, 4 figures

We study the problem of distinguishing between two distributions on a metric space; i.e., given metric measure spaces $({\mathbb X}, d, \mu_1)$ and $({\mathbb X}, d, \mu_2)$, we are interested in the problem of determining from finite data whether or not $\mu_1$ is $\mu_2$. The key is to use pairwise distances between observations and, employing a reconstruction theorem of Gromov, we can perform such a test using a two sample Kolmogorov--Smirnov test. A real analysis using phylogenetic trees and flu data is presented.

Mixture models are regularly used in density estimation applications, but the problem of estimating the mixing distribution remains a challenge. Nonparametric maximum likelihood produce estimates of the mixing distribution that are discrete, and these may be hard to interpret when the true mixing distribution is believed to have a smooth density. In this paper, we investigate an algorithm that produces a sequence of smooth estimates that has been conjectured to converge to the nonparametric maximum likelihood estimator. Here we give a rigorous proof of this conjecture, and propose a new data-driven stopping rule that produces smooth near-maximum likelihood estimates of the mixing density, and simulations demonstrate the quality empirical performance of this estimator., Comment: 11 pages, 3 figures

In this paper we use an iterative algorithm for solving Fredholm equations of the first kind. The basic algorithm is known and is based on an EM algorithm when involved functions are non-negative and integrable. With this algorithm we demonstrate two examples involving the estimation of a mixing density and a first passage time density function involving Brownian motion. We also develop the basic algorithm to include functions which are not necessarily non-negative and again present illustrations under this scenario. A self contained proof of convergence of all the algorithms employed is presented., Comment: 22 pages, 7 figures

Objective prior distributions represent an important tool that allows one to have the advantages of using the Bayesian framework even when information about the parameters of a model is not available. The usual objective approaches work off the chosen statistical model and in the majority of cases the resulting prior is improper, which can pose limitations to a practical implementation, even when the complexity of the model is moderate. In this paper we propose to take a novel look at the construction of objective prior distributions, where the connection with a chosen sampling distribution model is removed. We explore the notion of defining objective prior distributions which allow one to have some degree of flexibility, in particular in exhibiting some desirable features, such as being proper, or centered on specific values which would be of interest in nested model comparisons. The basic tool we use are proper scoring rules and the main result is a class of objective prior distributions that can be employed in scenarios where the usual model based priors fail, such as mixture models and model selection via Bayes factors. In addition, we show that the proposed class of priors is the result of minimising the information it contains, providing solid interpretation to the method.

A new approach on the joint estimation of partially exchangeable observations is presented by constructing pairwise dependence between $m$ random density functions, each of which is modeled as a mixture of geometric stick breaking processes. This approach is based on a new random central masses version of the Pairwise Dependent Dirichlet Process prior mixture model (PDDP) first introduced in Hatjispyros et al. (2011). The idea is to create pairwise dependence through random measures that are location-preserving-expectations of Dirichlet random measures. Our contention is that mixture modeling with Pairwise Dependent Geometric Stick Breaking Process (PDGSBP) priors is sufficient for prediction and estimation purposes; moreover the associated Gibbs sampler is much faster and easier to implement than its Dirichlet Process based counterpart. To this respect, we provide a-priori-synchronized comparison studies under sparse $m$-scalable synthetic and real data examples.

We propose a novel iterative algorithm for solving a large sparse linear system. The method is based on the EM algorithm. If the system has a unique solution, the algorithm guarantees convergence with a geometric rate. Otherwise, convergence to a minimal Kullback--Leibler divergence point is guaranteed. The algorithm is easy to code and competitive with other iterative algorithms., Comment: 18 pages, 3 figures, 1 table

Motivated by the fundamental problem of modeling the frequency of frequencies (FoF) distribution, this paper introduces the concept of a cluster structure to define a probability function that governs the joint distribution of a random count and its exchangeable random partitions. A cluster structure, naturally arising from a completely random measure mixed Poisson process, allows the probability distribution of the random partitions of a subset of a population to be dependent on the population size, a distinct and motivated feature that makes it more flexible than a partition structure. This allows it to model an entire FoF distribution whose structural properties change as the population size varies. A FoF vector can be simulated by drawing an infinite number of Poisson random variables, or by a stick-breaking construction with a finite random number of steps. A generalized negative binomial process model is proposed to generate a cluster structure, where in the prior the number of clusters is finite and Poisson distributed, and the cluster sizes follow a truncated negative binomial distribution. We propose a simple Gibbs sampling algorithm to extrapolate the FoF vector of a population given the FoF vector of a sample taken without replacement from the population. We illustrate our results and demonstrate the advantages of the proposed models through the analysis of real text, genomic, and survey data., Comment: To appear in the Journal of the American Statistical Association (Theory and Methods). 26 pages + 17 page supplement, 19 figures. arXiv admin note: text overlap with arXiv:1410.3155

In high-dimensional problems, choosing a prior distribution such that the corresponding posterior has desirable practical and theoretical properties can be challenging. This begs the question: can the data be used to help choose a good prior? In this paper, we develop a general strategy for constructing a data-driven or empirical prior and sufficient conditions for the corresponding posterior distribution to achieve a certain concentration rate. The idea is that the prior should put sufficient mass on parameter values for which the likelihood is large. An interesting byproduct of this data-driven centering is that the asymptotic properties of the posterior are less sensitive to the prior shape which, in turn, allows users to work with priors of computationally convenient forms while maintaining the desired rates. General results on both adaptive and non-adaptive rates based on empirical priors are presented, along with illustrations in density estimation, nonparametric regression, and high-dimensional structured normal models., Comment: 31 pages

In this paper, an approach to estimating a nonlinear deterministic model is presented. We introduce a stochastic model with extremely small variances so that the deterministic and stochastic models are essentially indistinguishable from each other. This point is explained in the paper. The estimation is then carried out using stochastic optimisation based on Markov chain Monte Carlo (MCMC) methods.

The aim of this paper is to find distributional results for the posterior parameters which arise in the Sethuraman (1994) representation of the Dirichlet process. These results can then be used to derive simply the posterior of the Dirichlet process.

The paper is concerned with constructing pairwise dependence between $m$ random density functions each of which is modeled as a mixture of Dirichlet process model. The key to this is how to create dependencies between random Dirichlet processes. The present paper adopts a plan previously used for creating pairwise dependence, with the simplification that all random Dirichlet processes share the same atoms. Our contention is that for all dependent Dirichlet process models, common atoms are sufficient. We show that by adopting common atoms, it is possible to compute the $L_p$ distances between all pairs of random probability measures.

A density estimation method in a Bayesian nonparametric framework is presented when recorded data are not coming directly from the distribution of interest, but from a length biased version. From a Bayesian perspective, efforts to computationally evaluate posterior quantities conditionally on length biased data were hindered by the inability to circumvent the problem of a normalizing constant. In this paper we present a novel Bayesian nonparametric approach to the length bias sampling problem which circumvents the issue of the normalizing constant. Numerical illustrations as well as a real data example are presented and the estimator is compared against its frequentist counterpart, the kernel density estimator for indirect data of Jones (1991).

We introduce a class of birth-and-death Polya urns, which allow for both sampling and removal of observations governed by an auxiliary inhomogeneous Bernoulli process, and investigate the asymptotic behaviour of the induced allelic partitions. By exploiting some embedded models, we show that the asymptotic regimes exhibit a phase transition from partitions with almost surely infinitely many blocks and independent counts, to stationary partitions with a random number of blocks. The first regime corresponds to limits of Ewens-type partitions and includes a result of Arratia, Barbour and Tavar\'e (1992) as a special case. We identify the invariant and reversible measure in the second regime, which preserves asymptotically the dependence between counts, and is shown to be a mixture of Ewens sampling formulas, with a tilted Negative Binomial mixing distribution on the sample size., Comment: Substantially revised

A Bayesian framework is attractive in the context of prediction, but a fast recursive update of the predictive distribution has apparently been out of reach, in part because Monte Carlo methods are generally used to compute the predictive. This paper shows that online Bayesian prediction is possible by characterizing the Bayesian predictive update in terms of a bivariate copula, making it unnecessary to pass through the posterior to update the predictive. In standard models, the Bayesian predictive update corresponds to familiar choices of copula but, in nonparametric problems, the appropriate copula may not have a closed-form expression. In such cases, our new perspective suggests a fast recursive approximation to the predictive density, in the spirit of Newton's predictive recursion algorithm, but without requiring evaluation of normalizing constants. Consistency of the new algorithm is shown, and numerical examples demonstrate its quality performance in finite-samples compared to fully Bayesian and kernel methods., Comment: 22 pages, 3 figures, 3 tables

In this paper we introduce a new class of multivariate unimodal distributions, motivated by Khintchine's representation. We start by proposing a univariate model, whose support covers all the unimodal distributions on the real line. The proposed class of unimodal distributions can be naturally extended to higher dimensions, by using the multivariate Gaussian copula. Under both univariate and multivariate settings, we provide MCMC algorithms to perform inference about the model parameters and predictive densities. The methodology is illustrated with univariate and bivariate examples, and with variables taken from a real data-set., Comment: 33 pages, 15 figures