Search

AbstractDuring an epidemic, the daily number of reported infected cases, deaths or hospitalizations is often lower than the actual number due to reporting delays. Nowcasting aims to estimate the cases that have not yet been reported and combine it with the already reported cases to obtain an estimate of the daily cases. In this paper, we present a fast and flexible Bayesian approach for nowcasting by combining P-splines and Laplace approximations. Laplacian-P-splines provide a flexible framework for nowcasting that is computationally less demanding as compared to traditional Markov chain Monte Carlo techniques. The proposed approach also permits to naturally quantify the prediction uncertainty. Model performance is assessed through simulations and the nowcasting method is applied to COVID-19 mortality and incidence cases in Belgium. [ABSTRACT FROM AUTHOR]

AbstractCount data with complex features arise in many disciplines, including ecology, agriculture, criminology, medicine, and public health. Zero inflation, spatial dependence, and non-equidispersion are common features in count data. There are currently two classes of models that allow for these features—the mode-parameterized Conway–Maxwell–Poisson (COMP) distribution and the generalized Poisson model. However both require the use of either constraints on the parameter space or a parameterization that leads to challenges in interpretability. We propose spatial mean-parameterized COMP models that retain the flexibility of these models while resolving the above issues. We use a Bayesian spatial filtering approach in order to efficiently handle high-dimensional spatial data and we use reversible-jump MCMC to automatically choose the basis vectors for spatial filtering. The COMP distribution poses two additional computational challenges—an intractable normalizing function in the likelihood and no closed-form expression for the mean. We propose a fast computational approach that addresses these challenges by, respectively, introducing an efficient auxiliary variable algorithm and pre-computing key approximations for fast likelihood evaluation. We illustrate the application of our methodology to simulated and real datasets, including Texas HPV-cancer data and US vaccine refusal data. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

AbstractApproximate Bayesian computation (ABC) is a class of Bayesian inference algorithms that targets problems with intractable or unavailable likelihood functions. It uses synthetic data drawn from the simulation model to approximate the posterior distribution. However, ABC is computationally intensive for complex models in which simulating synthetic data is very expensive. In this article, we propose an early rejection Markov chain Monte Carlo (ejMCMC) sampler based on Gaussian processes to accelerate inference speed. We early reject samples in the first stage of the kernel using a discrepancy model, in which the discrepancy between the simulated and observed data is modeled by Gaussian process (GP). Hence, synthetic data is generated only if the parameter space is worth exploring. We demonstrate through theory, simulation experiments, and real data analysis that the new algorithm significantly improves inference efficiency compared to existing early-rejection MCMC algorithms. In addition, we employ our proposed method within an ABC sequential Monte Carlo (SMC) sampler. In our numerical experiments, we use examples of ordinary differential equations, stochastic differential equations, and delay differential equations to demonstrate the effectiveness of the proposed algorithm. We develop an R package that is available at https://github.com/caofff/ejMCMC. [ABSTRACT FROM AUTHOR]

AbstractIn this paper, we propose combining the stochastic blockmodel and the smooth graphon model, two of the most prominent modeling approaches in statistical network analysis. Stochastic blockmodels are generally used for clustering networks into groups of actors with homogeneous connectivity behavior. Smooth graphon models, on the other hand, assume that all nodes can be arranged on a one-dimensional scale such that closeness implies a similar behavior in connectivity. Both frameworks belong to the class of node-specific latent variable models, entailing a natural relationship. While these two modeling concepts have developed independently, this paper proposes their generalization towards stochastic block smooth graphon models. This combined approach enables to exploit the advantages of both worlds. Employing concepts of the EM-type algorithm allows to develop an appropriate estimation routine, where MCMC techniques are used to accomplish the E-step. Simulations and real-world applications support the practicability of our novel method and demonstrate its advantages. The paper is accompanied by supplementary material covering details about computation and implementation. [ABSTRACT FROM AUTHOR]

AbstractComputer models play a crucial role in numerous scientific and engineering domains. To ensure the accuracy of simulations, it is essential to properly calibrate the input parameters of these models through statistical inference. While Bayesian inference is the standard approach for this task, employing Markov chain Monte Carlo methods often encounters computational hurdles due to the costly evaluation of likelihood functions and slow mixing rates. Although variational inference (VI) can be a fast alternative to traditional Bayesian approaches, VI has limited applicability due to boundary issues and local optima problems. To address these challenges, we propose flexible VI methods based on deep generative models that do not require parametric assumptions on the variational distribution. We embed a surjective transformation in our framework to avoid posterior truncation at the boundary. Additionally, we provide theoretical conditions that guarantee the success of the algorithm. Furthermore, our temperature annealing scheme and fine-tuning can prevent being trapped in local optima through a series of intermediate posteriors and weight adjustment. We apply our method to infectious disease models and a geophysical model, illustrating that the proposed method can provide fast and accurate inference compared to its competitors. [ABSTRACT FROM AUTHOR]

AbstractPosterior predictive p-values (ppps) have become popular tools for Bayesian model assessment, being general-purpose and easy to use. However, interpretation can be difficult because their distribution is not uniform under the hypothesis that the model did generate the data. Calibrated ppps (cppps) can be obtained via a bootstrap-like procedure, yet remain unavailable in practice due to high computational cost. This article introduces methods to enable efficient approximation of cppps and their uncertainty for fast model assessment. We first investigate the computational tradeoff between the number of calibration replicates and the number of MCMC samples per replicate. Provided that the MCMC chain from the real data has converged, using short MCMC chains per calibration replicate can save significant computation time compared to naive implementations, without significant loss in accuracy. We propose different variance estimators for the cppp approximation, which can be used to confirm quickly the lack of evidence against model misspecification. As variance estimation uses effective sample sizes of many short MCMC chains, we show these can be approximated well from the real-data MCMC chain. The procedure for cppp is implemented in NIMBLE, a flexible framework for hierarchical modeling that supports many models and discrepancy measures. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

AbstractMarkov chain Monte Carlo (MCMC) allows one to generate dependent replicates from a posterior distribution for effectively any Bayesian hierarchical model. However, MCMC can produce a significant computational burden. This motivates us to consider finding expressions of the posterior distribution that are computationally straightforward to obtain independent replicates from directly. We focus on a broad class of Bayesian hierarchical models for spatially dependent data, which are often modeled via a latent Gaussian process (LGP). First, we derive a new class of distributions referred to as the generalized conjugate multivariate (GCM) distribution. The GCM distribution’s theoretical development follows that of the conjugate multivariate (CM) distribution with two main differences: the GCM allows for latent Gaussian process assumptions, and the GCM explicitly accounts for hyperparameters through marginalization. The development of GCM is needed to obtain independent replicates directly from the exact posterior distribution, which has an efficient regression form. Hence, we refer to our method as Exact Posterior Regression (EPR). Simulation studies with weakly stationary spatial processes and spatial basis function expansions are provided. We provide an analysis of poverty incidence from the U.S. Census Bureau, and an analysis of high-dimensional remote sensing data. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

Sampling from matrix generalized inverse Gaussian (MGIG) distributions is required in Markov chain Monte Carlo (MCMC) algorithms for a variety of statistical models. However, an efficient sampling scheme for the MGIG distributions has not been fully developed. We here propose a novel blocked Gibbs sampler for the MGIG distributions based on the Cholesky decomposition. We show that the full conditionals of the entries of the diagonal and unit lower-triangular matrices are univariate generalized inverse Gaussian and multivariate normal distributions, respectively. Several variants of the Metropolis-Hastings algorithm can also be considered for this problem, but we mathematically prove that the average acceptance rates become extremely low in particular scenarios. We demonstrate the computational efficiency of the proposed Gibbs sampler through simulation studies and data analysis. for this article are available online. [ABSTRACT FROM AUTHOR]

Conditional particle filters (CPFs) with backward/ancestor sampling are powerful methods for sampling from the posterior distribution of the latent states of a dynamic model such as a hidden Markov model. However, the performance of these methods deteriorates with models involving weakly informative observations and/or slowly mixing dynamics. Both of these complications arise when sampling finely time-discretized continuous-time path integral models, but can occur with hidden Markov models too. Multinomial resampling, which is commonly employed with CPFs, resamples excessively for weakly informative observations and thereby introduces extra variance. Furthermore, slowly mixing dynamics render the backward/ancestor sampling steps ineffective, leading to degeneracy issues. We detail two conditional resampling strategies suitable for the weakly informative regime: the so-called "killing" resampling and the systematic resampling with mean partial order. To avoid the degeneracy issues, we introduce a generalization of the CPF with backward sampling that involves auxiliary "bridging" CPF steps that are parameterized by a blocking sequence. We present practical tuning strategies for choosing an appropriate blocking. Our experiments demonstrate that the CPF with a suitable resampling and the developed "bridge backward sampling" can lead to substantial efficiency gains in the weakly informative and slow mixing regime. for this article are available online. [ABSTRACT FROM AUTHOR]

We study the Bayesian multi-task variable selection problem, where the goal is to select activated variables for multiple related datasets simultaneously. We propose a new variational Bayes algorithm which generalizes and improves the recently developed "sum of single effects" model of Wang et al. Motivated by differential gene network analysis in biology, we further extend our method to joint structure learning of multiple directed acyclic graphical models, a problem known to be computationally highly challenging. We propose a novel order MCMC sampler where our multi-task variable selection algorithm is used to quickly evaluate the posterior probability of each ordering. Both simulation studies and real gene expression data analysis are conducted to show the efficiency of our method. Finally, we also prove a posterior consistency result for multi-task variable selection, which provides a theoretical guarantee for the proposed algorithms. for this article are available online. [ABSTRACT FROM AUTHOR]

Hidden Markov models (HMMs) are valuable tools for analyzing longitudinal data due to their capability to describe dynamic heterogeneity. Conventional HMMs typically assume that the number of hidden states (i.e., the order of HMMs) is known or predetermined through criterion-based methods. However, prior knowledge about the order is often unavailable, and a pairwise comparison using criterion-based methods becomes increasingly tedious and computationally demanding when the model space enlarges. A few studies have considered simultaneously performing order selection and parameter estimation under the frequentist framework. Still, they focused only on homogeneous HMMs and thus cannot accommodate situations where potential covariates affect the between-state transition. This study proposes a Bayesian double-penalized (BDP) procedure to conduct a simultaneous order selection and parameter estimation for heterogeneous HMMs. We develop a novel Markov chain Monte Carlo algorithm coupled with an efficient adjust-bound reversible jump scheme to address the challenges in updating the order. Simulation studies show that the proposed BDP procedure considerably outperforms the commonly used criterion-based methods. An application to the Alzheimer's Disease Neuroimaging Initiative study further confirms the utility of the proposed method. for this article are available online. [ABSTRACT FROM AUTHOR]

A prior distribution for the underlying graph is introduced in the framework of Gaussian graphical models. Such a prior distribution induces a block structure in the graph's adjacency matrix, allowing learning relationships between fixed groups of variables. A novel sampling strategy named Double Reversible Jumps Markov chain Monte Carlo is developed for learning block structured graphs under the conjugate G-Wishart prior. The algorithm proposes moves that add or remove not just a single edge of the graph but an entire group of edges. The method is then applied to smooth functional data. The classical smoothing procedure is improved by placing a graphical model on the basis expansion coefficients, providing an estimate of their conditional dependence structure. Since the elements of a B-Spline basis have compact support, the conditional dependence structure is reflected on well-defined portions of the domain. A known partition of the functional domain is exploited to investigate relationships among portions of the domain and improve the interpretability of the results. for this article are available online. [ABSTRACT FROM AUTHOR]

Feature allocation models postulate a sampling distribution whose parameters are derived from shared features. Bayesian models place a prior distribution on the feature allocation, and Markov chain Monte Carlo is typically used for model fitting, which results in thousands of feature allocations sampled from the posterior distribution. Based on these samples, we propose a method to provide a point estimate of a latent feature allocation. First, we introduce FARO loss, a function between feature allocations which satisfies quasi-metric properties and allows for comparing feature allocations with differing numbers of features. The loss involves finding the optimal feature ordering among all possible orderings, but computational feasibility is achieved by framing this task as a linear assignment problem. We also introduce the FANGS algorithm to obtain a Bayes estimate by minimizing the Monte Carlo estimate of the posterior expected FARO loss using the available samples. FANGS can produce an estimate other than those visited in the Markov chain. We provide an investigation of existing methods and our proposed methods. Our loss function and search algorithm are implemented in the fangs package in R. [ABSTRACT FROM AUTHOR]

Recently nonreversible samplers based on simulating piecewise deterministic Markov processes (PDMPs) have shown potential for efficient sampling in Bayesian inference problems. However, there remains a lack of guidance on how to best implement these algorithms. If implemented poorly, the computational costs of simulating event times can outweigh the statistical efficiency of the nonreversible dynamics. Drawing on the adaptive rejection literature, we propose the concave-convex adaptive thinning approach for simulating a piecewise deterministic Markov process, which we call CC-PDMP. This approach provides a general guide for constructing bounds that may be used to facilitate PDMP-based sampling. A key advantage of this method is its additive structure—adding concave-convex decompositions yields a concave-convex decomposition. This makes the construction of bounds modular, as given a concave-convex decomposition for a class of likelihoods and a family of priors, they can be combined to construct bounds for the posterior. We show that constructing our bounds is simple and leads to computationally efficient thinning. Our approach is well suited to local PDMP simulation where conditional independence of the target can be exploited for potentially huge computational gains. We provide an R package and compare with existing approaches to simulating events in the PDMP literature. for this article are available online. [ABSTRACT FROM AUTHOR]

We propose a novel hybrid quantum computing strategy for parallel MCMC algorithms that generate multiple proposals at each step. This strategy makes the rate-limiting step within parallel MCMC amenable to quantum parallelization by using the Gumbel-max trick to turn the generalized accept-reject step into a discrete optimization problem. When combined with new insights from the parallel MCMC literature, such an approach allows us to embed target density evaluations within a well-known extension of Grover's quantum search algorithm. Letting P denote the number of proposals in a single MCMC iteration, the combined strategy reduces the number of target evaluations required from O (P) to O (P) . In the following, we review the rudiments of quantum computing, quantum search and the Gumbel-max trick in order to elucidate their combination for as wide a readership as possible. [ABSTRACT FROM AUTHOR]

Among Markov chain Monte Carlo algorithms, Hamiltonian Monte Carlo (HMC) is often the algorithm of choice for complex, high-dimensional target distributions; however, its efficiency is notoriously sensitive to the choice of the integration-time tuning parameter. When integrating both forward and backward in time using the same leapfrog integration step as HMC, the set of apogees, local maxima in the potential along a path, is the same whatever point (position and momentum) along the path is chosen to initialize the integration. We present the Apogee to Apogee Path Sampler (AAPS), which uses this invariance to create a simple yet generic methodology for constructing a path, proposing a point from it and accepting or rejecting that proposal so as to target the intended distribution. We demonstrate empirically that AAPS has a similar efficiency to HMC but is much more robust to the setting of its equivalent tuning parameter, the number of apogees that the path crosses. for this article are available online. [ABSTRACT FROM AUTHOR]

The seminal Bradley-Terry model exhibits transitivity, that is, the property that the probabilities of player A beating B and B beating C give the probability of A beating C, with these probabilities determined by a skill parameter for each player. Such transitive models do not account for different strategies of play between each pair of players, which gives rise to intransitivity. Various intransitive parametric models have been proposed but they lack the flexibility to cover the different strategies across n players, with the O (n 2) values of intransitivity modeled using O (n) parameters, while they are not parsimonious when the intransitivity is simple. We overcome their lack of adaptability by allocating each pair of players to one of a random number of K intransitivity levels, each level representing a different strategy. Our novel approach for the skill parameters involves having the n players allocated to a random number of A < n distinct skill levels, to improve efficiency and avoid false rankings. Although we may have to estimate up to O (n 2) unknown parameters for (A , K) we anticipate that in many practical contexts A + K < n . Our semiparametric model, which gives the Bradley-Terry model when (A = n − 1 , K = 0) , is shown to have an improved fit relative to the Bradley-Terry, and the existing intransitivity models, in out-of-sample testing when applied to simulated and American League baseball data. for the article areavailable online. [ABSTRACT FROM AUTHOR]

In recent years, conditional copulas, that allow dependence between variables to vary according to the values of one or more covariates, have attracted increasing attention. However, the literature mainly focused on the bivariate case, since the constraints on the multivariate copulas correlation matrices would make the specifications of covariates arduous. In high dimension, vine copulas offer greater flexibility compared to multivariate copulas, since they are constructed using bivariate copulas as building blocks. We present a novel inferential approach for multivariate distributions, which combines the flexibility of vine constructions with the advantages of Bayesian nonparametrics, not requiring the specification of parametric families for each pair copula. Expressing multivariate copulas using vines allows us to easily account for covariate specifications driving the dependence between response variables. We specify the vine copula density as an infinite mixture of Gaussian copulas, defining a Dirichlet process prior on the mixing measure, and performing posterior inference via Markov chain Monte Carlo sampling. Our approach is successful as for clustering as well as for density estimation. We carry out simulation studies and apply the proposed approach to analyze a veterinary dataset and to investigate the impact of natural disasters on financial development. for this article are available online. [ABSTRACT FROM AUTHOR]

Proximal Markov chain Monte Carlo is a novel construct that lies at the intersection of Bayesian computation and convex optimization, which helped popularize the use of nondifferentiable priors in Bayesian statistics. Existing formulations of proximal MCMC, however, require hyperparameters and regularization parameters to be prespecified. In this article, we extend the paradigm of proximal MCMC through introducing a novel new class of nondifferentiable priors called epigraph priors. As a proof of concept, we place trend filtering, which was originally a nonparametric regression problem, in a parametric setting to provide a posterior median fit along with credible intervals as measures of uncertainty. The key idea is to replace the nonsmooth term in the posterior density with its Moreau-Yosida envelope, which enables the application of the gradient-based MCMC sampler Hamiltonian Monte Carlo. The proposed method identifies the appropriate amount of smoothing in a data-driven way, thereby automating regularization parameter selection. Compared with conventional proximal MCMC methods, our method is mostly tuning free, achieving simultaneous calibration of the mean, scale and regularization parameters in a fully Bayesian framework. for this article are available online. [ABSTRACT FROM AUTHOR]

In this work, we propose a novel approximated collapsed variational Bayes approach to model selection in linear regression. The approximated collapsed variational Bayes algorithm offers improvements over mean field variational Bayes by marginalizing over a subset of parameters and using mean field variational Bayes over the remaining parameters in an analogous fashion to collapsed Gibbs sampling. We have shown that the proposed algorithm, under typical regularity assumptions, (a) includes variables in the true underlying model at an exponential rate in the sample size, or (b) excludes the variables at least at the first order rate in the sample size if the variables are not in the true model. Simulation studies show that the performance of the proposed method is close to that of a particular Markov chain Monte Carlo sampler and a path search based variational Bayes algorithm, but requires an order of magnitude less time. The proposed method is also highly competitive with penalized methods, expectation propagation, stepwise AIC/BIC, BMS, and EMVS under various settings. for the article are available online. [ABSTRACT FROM AUTHOR]

Data augmentation (DA) algorithms are slow in massive data settings due to multiple passes through the entire data. We address this problem by developing a DA extension that exploits asynchronous and distributed computing. The extended DA algorithm is called Asynchronous and Distributed (AD) DA with the original DA as its parent. Any ADDA is indexed by a parameter r ∈ (0 , 1) and starts by dividing the entire data into k disjoint subsets and storing them on k processes. Every iteration of ADDA augments only an r-fraction of the k data subsets with some positive probability and leaves the remaining (1 − r) -fraction of the augmented data unchanged. The parameter draws are obtained using the r-fraction of new and (1 − r) -fraction of old augmented data. We show that the ADDA Markov chain is Harris ergodic with the desired stationary distribution under mild conditions on the parent DA algorithm. We demonstrate that ADDA is significantly faster than its parent for many (k, r) choices in three representative models. We also establish the geometric ergodicity of the ADDA Markov chain for all the three models, which yields asymptotically valid standard errors for estimates of desired posterior quantities. for this article are available online. [ABSTRACT FROM AUTHOR]

In this article, we introduce a new and efficient data augmentation approach to the posterior inference of the models with shape parameters when the reciprocal gamma function appears in full conditional densities. Our approach is to approximate full conditional densities of shape parameters by using Gauss's multiplication formula and Stirling's formula for the gamma function, where the approximation error can be made arbitrarily small. We use the techniques to construct efficient Gibbs and Metropolis–Hastings algorithms for a variety of models that involve the gamma distribution, Student's t-distribution, the Dirichlet distribution, the negative binomial distribution, and the Wishart distribution. The proposed sampling method is numerically demonstrated through simulation studies. for this article are available online. [ABSTRACT FROM AUTHOR]

Key to effective generic, or "black-box," variational inference is the selection of an approximation to the target density that balances accuracy and speed. Copula models are promising options, but calibration of the approximation can be slow for some choices. Smith, Loaiza-Maya, and Nott (2020) suggest using tractable and scalable "implicit copula" models that are formed by element-wise transformation of the target parameters. We propose an adjustment to these transformations that make the approximation invariant to the scale and location of the target density. We also show how a sub-class of elliptical copulas have a generative representation that allows easy application of the re-parameterization trick and efficient first order optimization. We demonstrate the estimation methodology using two statistical models as examples. The first is a mixed effects logistic regression, and the second is a regularized correlation matrix. For the latter, standard Markov chain Monte Carlo estimation methods can be slow or difficult to implement, yet our proposed variational approach provides an effective and scalable estimator. We illustrate by estimating a regularized Gaussian copula model for income inequality in U.S. states between 1917 and 2018. An Online Appendix and MATLAB code to implement the method are available as . [ABSTRACT FROM AUTHOR]

We establish conditions under which Metropolis-Hastings (MH) algorithms with a position-dependent proposal covariance matrix will or will not have the geometric rate of convergence. Some of the diffusions based MH algorithms like the Metropolis adjusted Langevin algorithm (MALA) and the pre-conditioned MALA (PCMALA) have a position-independent proposal variance. Whereas, for other modern variants of MALA like the manifold MALA (MMALA) that adapt to the geometry of the target distributions, the proposal covariance matrix changes in every iteration. Thus, we provide conditions for geometric ergodicity of different variations of the Langevin algorithms. These results have important practical implications as these provide crucial justification for the use of asymptotically valid Monte Carlo standard errors for Markov chain based estimates. The general conditions are verified in the context of conditional simulation from the two most popular generalized linear mixed models (GLMMs), namely the binomial GLMM with the logit link and the Poisson GLMM with the log link. Empirical comparison in the framework of some spatial GLMMs shows that the computationally less expensive PCMALA with an appropriately chosen pre-conditioning matrix may outperform the MMALA. [ABSTRACT FROM AUTHOR]

Bayesian inference for Continuous-Time Markov chains (CTMCs) on countably infinite spaces is notoriously difficult because evaluating the likelihood exactly is intractable. One way to address this challenge is to first build a nonnegative and unbiased estimate of the likelihood—involving the matrix exponential of finite truncations of the true rate matrix—and then to use the estimates in a pseudo-marginal inference method. In this work, we show that we can dramatically increase the efficiency of this approach by avoiding the computation of exact matrix exponentials. In particular, we develop a general methodology for constructing an unbiased, nonnegative estimate of the likelihood using doubly-monotone matrix exponential approximations. We further develop a novel approximation in this family—the skeletoid—as well as theory regarding its approximation error and how that relates to the variance of the estimates used in pseudo-marginal inference. Experimental results show that our approach yields more efficient posterior inference for a wide variety of CTMCs. for this article are available online. [ABSTRACT FROM AUTHOR]

While Bayesian functional mixed models have been shown effective to model functional data with various complex structures, their application to extremely high-dimensional data is limited due to computational challenges involved in posterior sampling. We introduce a new computational framework that enables ultra-fast approximate inference for high-dimensional data in functional form. This framework adopts parsimonious basis to represent functional observations, which facilitates efficient compression and parallel computing in basis space. Instead of performing expensive Markov chain Monte Carlo sampling, we approximate the posterior distribution using variational Bayes and adopt a fast iterative algorithm to estimate parameters of the approximate distribution. Our approach facilitates a fast multiple testing procedure in basis space, which can be used to identify significant local regions that reflect differences across groups of samples. We perform two simulation studies to assess the performance of approximate inference, and demonstrate applications of the proposed approach by using a proteomic mass spectrometry dataset and a brain imaging dataset. for this article are available online. [ABSTRACT FROM AUTHOR]

Inference for high-dimensional, large scale and long series dynamic systems is a challenging task in modern data science. The existing algorithms, such as particle filter or sequential importance sampler, do not scale well to the dimension of the system and the sample size of the dataset, and often suffers from the sample degeneracy issue for long series data. The recently proposed Langevinized ensemble Kalman filter (LEnKF) addresses these difficulties in a coherent way. However, it cannot be applied to the case that the dynamic system contains unknown parameters. This article proposes the so-called stochastic approximation-LEnKF for jointly estimating the states and unknown parameters of the dynamic system, where the parameters are estimated on the fly based on the state variables simulated by the LEnKF under the framework of stochastic approximation Markov chain Monte Carlo (MCMC). Under mild conditions, we prove its consistency in parameter estimation and ergodicity in state variable simulations. The proposed algorithm can be used in uncertainty quantification for long series, large scale, and high-dimensional dynamic systems. Numerical results indicate its superiority over the existing algorithms. We employ the proposed algorithm in state-space modeling of the sea surface temperature with a long short term memory (LSTM) network, which indicates its great potential in statistical analysis of complex dynamic systems encountered in modern data science. for this article are available online. [ABSTRACT FROM AUTHOR]

Existing changepoint approaches consider changepoints to occur linearly in time; one changepoint happens after another and they are not linked. However, data processes may have regularly occurring changepoints, for example, a yearly increase in sales of ice-cream on the first hot weekend. Using linear changepoint approaches here will miss more global features such as a decrease in sales of ice-cream due to other product availability. Being able to tease these global changepoint features from the more local (periodic) ones is beneficial for inference. We propose a periodic changepoint model to model this behavior using a mixture of a periodic and linear time perspective. Built around a Reversible Jump Markov chain Monte Carlo sampler, the Bayesian framework is used to study the local (periodic) changepoint behavior. To identify the optimal global changepoint positions we integrate the local changepoint model into the pruned exact linear time (PELT) search algorithm. We demonstrate that the method detects both local and global changepoints with high accuracy on simulated and motivating applications that share periodic behavior. Due to the micro–macro nature of the analysis, visualization of the results can be challenging. We additionally provide a unique perspective for changepoint visualizations in these data sequences. for this article are available online. [ABSTRACT FROM AUTHOR]

This article considers a joint modeling framework for simultaneously examining the dynamic pattern of longitudinal and ultrahigh-dimensional images and their effects on the survival of interest. A functional mixed effects model is considered to describe the trajectories of longitudinal images. Then, a high-dimensional functional principal component analysis (HD-FPCA) is adopted to extract the principal eigenimages to reduce the ultrahigh dimensionality of imaging data. Finally, a Cox regression model is used to examine the effects of the longitudinal images and other risk factors on the hazard. A theoretical justification shows that a naive two-stage procedure that separately analyzes each part of the joint model produces biased estimation even if the longitudinal images have no measurement error. We develop a Bayesian joint estimation method coupled with efficient Markov chain Monte Carlo sampling schemes to perform statistical inference for the proposed joint model. A Monte Carlo dynamic prediction procedure is proposed to predict the future survival probabilities of subjects given their historical longitudinal images. The proposed model is assessed through extensive simulation studies and an application to Alzheimer's Disease Neuroimaging Initiative, which turns out to hold the promise of accuracy and possess higher predictive capacity for survival outcome compared with existing methods. for this article are available online. [ABSTRACT FROM AUTHOR]

This article proposes a new methodology to perform Bayesian inference for a class of multidimensional Cox processes in which the intensity function is piecewise constant. Poisson processes with piecewise constant intensity functions are believed to be suitable to model a variety of point process phenomena and, given its simpler structure, are expected to provide more precise inference when compared to processes with nonparametric and continuously varying intensity functions. The partition of the space domain is flexibly determined by a level-set function of a latent Gaussian process. Despite the intractability of the likelihood function and the infinite dimensionality of the parameter space, inference is performed exactly, in the sense that no space discretization approximation is used and MCMC error is the only source of inaccuracy. That is achieved by using retrospective sampling techniques and devising a pseudo-marginal infinite-dimensional MCMC algorithm that converges to the exact target posterior distribution. Computational efficiency is favored by considering a nearest neighbor Gaussian process, allowing for the analysis of large datasets. An extension to consider spatiotemporal models is also proposed. The efficiency of the proposed methodology is investigated in simulated examples and its applicability is illustrated in the analysis of some real point process datasets. [ABSTRACT FROM AUTHOR]

We define a novel class of additive models, called Extended Latent Gaussian Models, that allow for a wide range of response distributions and flexible relationships between the additive predictor and mean response. The new class covers a broad range of interesting models including multi-resolution spatial processes, partial likelihood-based survival models, and multivariate measurement error models. Because computation of the exact posterior distribution is infeasible, we develop a fast, scalable approximate Bayesian inference methodology for this class based on nested Gaussian, Laplace, and adaptive quadrature approximations. We prove that the error in these approximate posteriors is o p (1) under standard conditions, and provide numerical evidence suggesting that our method runs faster and scales to larger datasets than methods based on Integrated Nested Laplace Approximations and Markov chain Monte Carlo, with comparable accuracy. We apply the new method to the mapping of malaria incidence rates in continuous space using aggregated data, mapping leukemia survival hazards using a Cox Proportional-Hazards model with a continuously-varying spatial process, and estimating the mass of the Milky Way Galaxy using noisy multivariate measurements of the positions and velocities of star clusters in its orbit. for this article are available online. [ABSTRACT FROM AUTHOR]

We present new methodologies for Bayesian inference on the rate parameters of a discretely observed continuous-time Markov jump process with a countably infinite statespace. The usual method of choice for inference, particle Markov chain Monte Carlo (particle MCMC), struggles when the observation noise is small. We consider the most challenging regime of exact observations and provide two new methodologies for inference in this case: the minimal extended statespace algorithm (MESA) and the nearly minimal extended statespace algorithm (nMESA). By extending the Markov chain Monte Carlo statespace, both MESA and nMESA use the exponentiation of finite rate matrices to perform exact Bayesian inference on the Markov jump process even though its statespace is countably infinite. Numerical experiments show improvements over particle MCMC of between a factor of three and several orders of magnitude. for this article are available online. [ABSTRACT FROM AUTHOR]

We propose a Bayesian nonparametric matrix clustering approach to analyze the latent heterogeneity structure in the shot selection data collected from professional basketball players in the National Basketball Association (NBA). The proposed method adopts a mixture of finite mixtures framework and fully uses the spatial information via a mixture of matrix normal distribution representation. We propose an efficient Markov chain Monte Carlo algorithm for posterior sampling that allows simultaneous inference on both the number of clusters and the cluster configurations. We also establish large-sample convergence properties for the posterior distribution. The compelling empirical performance of the proposed method is demonstrated via simulation studies and an application to shot chart data from selected players in the NBAs 2017–2018 regular season. for this article are available online. [ABSTRACT FROM AUTHOR]

We propose a novel efficient model-fitting algorithm for state space models. State space models are an intuitive and flexible class of models, frequently used due to the combination of their natural separation of the different mechanisms acting on the system of interest: the latent underlying system process; and the observation process. This flexibility, however, often comes at the price of more complicated model-fitting algorithms due to the associated analytically intractable likelihood. For the general case a Bayesian data augmentation approach is often employed, where the true unknown states are treated as auxiliary variables and imputed within the MCMC algorithm. However, standard "vanilla" MCMC algorithms may perform very poorly due to high correlation between the imputed states and/or parameters, often leading to model-specific bespoke algorithms being developed that are nontransferable to alternative models. The proposed method addresses the inefficiencies of traditional approaches by combining data augmentation with numerical integration in a Bayesian hybrid approach. This approach permits the use of standard "vanilla" updating algorithms that perform considerably better than the traditional approach in terms of improved mixing and lower autocorrelation, and has the potential to be incorporated into bespoke model-specific algorithms. To demonstrate the ideas, we apply our semi-complete data augmentation algorithm to different application areas and models, leading to distinct implementation schemes and improved mixing and demonstrating improved mixing of the model parameters. for this article are available online. [ABSTRACT FROM AUTHOR]

Distance weighted discrimination (DWD) is a linear discrimination method that is particularly well-suited for classification tasks with high-dimensional data. The DWD coefficients minimize an intuitive objective function, which can solved efficiently using state-of-the-art optimization techniques. However, DWD has not yet been cast into a model-based framework for statistical inference. In this article we show that DWD identifies the mode of a proper Bayesian posterior distribution, that results from a particular link function for the class probabilities and a shrinkage-inducing proper prior distribution on the coefficients. We describe a relatively efficient Markov chain Monte Carlo (MCMC) algorithm to simulate from the true posterior under this Bayesian framework. We show that the posterior is asymptotically normal and derive the mean and covariance matrix of its limiting distribution. Through several simulation studies and an application to breast cancer genomics we demonstrate how the Bayesian approach to DWD can be used to (a) compute well-calibrated posterior class probabilities, (b) assess uncertainty in the DWD coefficients and resulting sample scores, (c) improve power via semisupervised analysis when not all class labels are available, and (d) automatically determine a penalty tuning parameter within the model-based framework. R code to perform Bayesian DWD is available at . for this article are available online. [ABSTRACT FROM AUTHOR]

Gaussian graphical models can capture complex dependency structures among variables. For such models, Bayesian inference is attractive as it provides principled ways to incorporate prior information and to quantify uncertainty through the posterior distribution. However, posterior computation under the conjugate G-Wishart prior distribution on the precision matrix is expensive for general nondecomposable graphs. We therefore propose a new Markov chain Monte Carlo (MCMC) method named the G-Wishart weighted proposal algorithm (WWA). WWA's distinctive features include delayed acceptance MCMC, Gibbs updates for the precision matrix and an informed proposal distribution on the graph space that enables embarrassingly parallel computations. Compared to existing approaches, WWA reduces the frequency of the relatively expensive sampling from the G-Wishart distribution. This results in faster MCMC convergence, improved MCMC mixing and reduced computing time. Numerical studies on simulated and real data show that WWA provides a more efficient tool for posterior inference than competing state-of-the-art MCMC algorithms. Supplemental materials for the article are available online. [ABSTRACT FROM AUTHOR]

Numerical generalized randomized Hamiltonian Monte Carlo is introduced, as a robust, easy to use and computationally fast alternative to conventional Markov chain Monte Carlo methods for continuous target distributions. A wide class of piecewise deterministic Markov processes generalizing Randomized HMC (Bou-Rabee and Sanz-Serna) by allowing for state-dependent event rates is defined. Under very mild restrictions, such processes will have the desired target distribution as an invariant distribution. Second, the numerical implementation of such processes, based on adaptive numerical integration of second order ordinary differential equations (ODEs) is considered. The numerical implementation yields an approximate, yet highly robust algorithm that, unlike conventional Hamiltonian Monte Carlo, enables the exploitation of the complete Hamiltonian trajectories (hence, the title). The proposed algorithm may yield large speedups and improvements in stability relative to relevant benchmarks, while incurring numerical biases that are negligible relative to the overall Monte Carlo errors. Granted access to a high-quality ODE code, the proposed methodology is both easy to implement and use, even for highly challenging and high-dimensional target distributions. for this article are available online. [ABSTRACT FROM AUTHOR]

Estimating the marginal and joint densities of the long-term average intakes of different dietary components is an important problem in nutritional epidemiology. Since these variables cannot be directly measured, data are usually collected in the form of 24-hour recalls of the intakes. The problem of estimating the density of the latent long-term average intakes from their observed but error contaminated recalls then becomes a problem of multivariate deconvolution of densities. The underlying densities could potentially vary with the subjects' demographic characteristics such as sex, ethnicity, age, etc. The problem of density deconvolution in the presence of associated precisely measured covariates has, however, never been considered before, not even in the univariate setting. We present a flexible Bayesian semiparametric approach to covariate informed multivariate deconvolution. Building on recent advances on copula deconvolution and conditional tensor factorization techniques, our proposed method not only allows the joint and the marginal densities to vary flexibly with the associated predictors but also allows automatic selection of the most influential predictors. Importantly, the method also allows the density of interest and the density of the measurement errors to vary with potentially different sets of predictors. We design Markov chain Monte Carlo algorithms that enable efficient posterior inference, appropriately accommodating uncertainty in all aspects of our analysis. The empirical efficacy of the proposed method is illustrated through simulation experiments. Its practical utility is demonstrated in the afore-described nutritional epidemiology applications in estimating covariate adjusted long term intakes of different dietary components. An important by-product of the approach is a solution to covariate informed ordinary multivariate density estimation. include substantive additional details and R codes are also available online. [ABSTRACT FROM AUTHOR]

We introduce a methodology for online estimation of smoothing expectations for a class of additive functionals, in the context of a rich family of diffusion processes (that may include jumps) – observed at discrete-time instances. We overcome the unavailability of the transition density of the underlying SDE by working on the augmented pathspace. The new method can be applied, for instance, to carry out online parameter inference for the designated class of models. Algorithms defined on the infinite-dimensional pathspace have been developed the last years mainly in the context of MCMC techniques. There, the main benefit is the achievement of mesh-free mixing times for the practical time-discretised algorithm used on a PC. Our own methodology sets up the framework for infinite-dimensional online filtering – an important positive practical consequence is the construct of estimates with variance that does not increase with decreasing mesh-size. Besides regularity conditions, our method is, in principle, applicable under the weak assumption – relatively to restrictive conditions often required in the MCMC or filtering literature of methods defined on pathspace – that the SDE covariance matrix is invertible. [ABSTRACT FROM AUTHOR]

Autocovariances are a fundamental quantity of interest in Markov chain Monte Carlo (MCMC) simulations with autocorrelation function (ACF) plots being an integral visualization tool for performance assessment. Unfortunately, for slow-mixing Markov chains, the empirical autocovariance can highly underestimate the truth. For multiple-chain MCMC sampling, we propose a globally centered estimator of the autocovariance function (G-ACvF) that exhibits significant theoretical and empirical improvements. We show that the bias of the G-ACvF estimator is smaller than the bias of the current state-of-the-art. The impact of this improved estimator is evident in three critical output analysis applications: (a) ACF plots, (b) estimates of the Monte Carlo asymptotic covariance matrix, and (c) estimates of the effective sample size. Under weak conditions, we establish strong consistency of our improved asymptotic covariance estimator, and obtain its large-sample bias and variance. The performance of the new estimators is demonstrated through various examples. for this article are available online. [ABSTRACT FROM AUTHOR]

We study Markov chain Monte Carlo (MCMC) algorithms for target distributions defined on matrix spaces. Such an important sampling problem has yet to be analytically explored. We carry out a major step in covering this gap by developing the proper theoretical framework that allows for the identification of ergodicity properties of typical MCMC algorithms, relevant in such a context. Beyond the standard Random-Walk Metropolis (RWM) and preconditioned Crank–Nicolson (pCN), a contribution of this article in the development of a novel algorithm, termed the "Mixed" pCN (MpCN). RWM and pCN are shown not to be geometrically ergodic for an important class of matrix distributions with heavy tails. In contrast, MpCN is robust across targets with different tail behavior and has very good empirical performance within the class of heavy-tailed distributions. Geometric ergodicity for MpCN is not fully proven in this work, as some remaining drift conditions are quite challenging to obtain owing to the complexity of the state space. We do, however, make a lot of progress toward a proof, and show in detail the last steps left for future work. We illustrate the computational performance of the various algorithms through numerical applications, including calibration on real data of a challenging model arising in financial statistics. for this article are available online. [ABSTRACT FROM AUTHOR]

Bayesian networks are probabilistic graphical models widely employed to understand dependencies in high-dimensional data, and even to facilitate causal discovery. Learning the underlying network structure, which is encoded as a directed acyclic graph (DAG) is highly challenging mainly due to the vast number of possible networks in combination with the acyclicity constraint. Efforts have focused on two fronts: constraint-based methods that perform conditional independence tests to exclude edges and score and search approaches which explore the DAG space with greedy or MCMC schemes. Here, we synthesize these two fields in a novel hybrid method which reduces the complexity of MCMC approaches to that of a constraint-based method. Individual steps in the MCMC scheme only require simple table lookups so that very long chains can be efficiently obtained. Furthermore, the scheme includes an iterative procedure to correct for errors from the conditional independence tests. The algorithm offers markedly superior performance to alternatives, particularly because it also offers the possibility to sample DAGs from their posterior distribution, enabling full Bayesian model averaging for much larger Bayesian networks. for this article are available online. [ABSTRACT FROM AUTHOR]

Predictive models for binary data are fundamental in various fields, and the growing complexity of modern applications has motivated several flexible specifications for modeling the relationship between the observed predictors and the binary responses. A widely-implemented solution is to express the probability parameter via a probit mapping of a Gaussian process indexed by predictors. However, unlike for continuous settings, there is a lack of closed-form results for predictive distributions in binary models with Gaussian process priors. Markov chain Monte Carlo methods and approximation strategies provide common solutions to this problem, but state-of-the-art algorithms are either computationally intractable or inaccurate in moderate-to-high dimensions. In this article, we aim to cover this gap by deriving closed-form expressions for the predictive probabilities in probit Gaussian processes that rely either on cumulative distribution functions of multivariate Gaussians or on functionals of multivariate truncated normals. To evaluate these quantities we develop novel scalable solutions based on tile-low-rank Monte Carlo methods for computing multivariate Gaussian probabilities, and on mean-field variational approximations of multivariate truncated normals. Closed-form expressions for the marginal likelihood and for the posterior distribution of the Gaussian process are also discussed. As shown in simulated and real-world empirical studies, the proposed methods scale to dimensions where state-of-the-art solutions are impractical. [ABSTRACT FROM AUTHOR]

We construct a zig-zag process targeting a posterior distribution defined on a hybrid state space consisting of both discrete and continuous variables. The construction does not require any assumptions on the structure among discrete variables. We demonstrate our method on two examples in genetics based on the Kingman coalescent, showing that the zig-zag process can lead to efficiency gains of up to several orders of magnitude over classical Metropolis–Hastings algorithms, and that it is well suited to parallel computation. Our construction resembles existing techniques for Hamiltonian Monte Carlo on a hybrid state space, which suffers from implementationally and analytically complex boundary crossings when applied to the coalescent. We demonstrate that the continuous-time zig-zag process avoids these complications. for this article are available online. [ABSTRACT FROM AUTHOR]

We present the AdaptSPEC-X method for the joint analysis of a panel of possibly nonstationary time series. The approach is Bayesian and uses a covariate-dependent infinite mixture model to incorporate multiple time series, with mixture components parameterized by a time-varying mean and log spectrum. The mixture components are based on AdaptSPEC, a nonparametric model which adaptively divides the time series into an unknown number of segments and estimates the local log spectra by smoothing splines. AdaptSPEC-X extends AdaptSPEC in three ways. First, through the infinite mixture, it applies to multiple time series linked by covariates. Second, it can handle missing values, a common feature of time series which can cause difficulties for nonparametric spectral methods. Third, it allows for a time-varying mean. Through these extensions, AdaptSPEC-X can estimate time-varying means and spectra at observed and unobserved covariate values, allowing for predictive inference. Estimation is performed by Markov chain Monte Carlo (MCMC) methods, combining data augmentation, reversible jump, and Riemann manifold Hamiltonian Monte Carlo techniques. We evaluate the methodology using simulated data, and describe applications to Australian rainfall data and measles incidence in the United States. Software implementing the method proposed in this article is available in the R package BayesSpec. Supplementary files for this article are available online. [ABSTRACT FROM AUTHOR]

Repulsive mixture models have recently gained popularity for Bayesian cluster detection. Compared to more traditional mixture models, repulsive mixture models produce a smaller number of well-separated clusters. The most commonly used methods for posterior inference either require to fix a priori the number of components or are based on reversible jump MCMC computation. We present a general framework for mixture models, when the prior of the "cluster centers" is a finite repulsive point process depending on a hyperparameter, specified by a density which may depend on an intractable normalizing constant. By investigating the posterior characterization of this class of mixture models, we derive a MCMC algorithm which avoids the well-known difficulties associated to reversible jump MCMC computation. In particular, we use an ancillary variable method, which eliminates the problem of having intractable normalizing constants in the Hastings ratio. The ancillary variable method relies on a perfect simulation algorithm, and we demonstrate this is fast because the number of components is typically small. In several simulation studies and an application on sociological data, we illustrate the advantage of our new methodology over existing methods, and we compare the use of a determinantal or a repulsive Gibbs point process prior model. Supplementary files for this article are available online. [ABSTRACT FROM AUTHOR]

Longitudinal item response data are common in social science, educational science, and psychology, among other disciplines. Studying the time-varying relationships between items is crucial for educational assessment or designing marketing strategies from survey questions. Although dynamic network models have been widely developed, we cannot apply them directly to item response data because there are multiple systems of nodes with various types of local interactions among items, resulting in multiplex network structures. We propose a new model to study these temporal interactions among items by embedding the functional parameters within the exponential random graph model framework. Inference on such models is difficult because the likelihood functions contain intractable normalizing constants. Furthermore, the number of functional parameters grows exponentially as the number of items increases. Variable selection for such models is not trivial because standard shrinkage approaches do not consider temporal trends in functional parameters. To overcome these challenges, we develop a novel Bayes approach by combining an auxiliary variable MCMC algorithm and a recently developed functional shrinkage method. We apply our algorithm to survey and review datasets, illustrating that the proposed approach can avoid the evaluation of intractable normalizing constants as well as the detection of significant temporal interactions among items. Through a simulation study under different scenarios, we examine the performance of our algorithm. Our method is, to our knowledge, the first attempt to select functional variables for models with intractable normalizing constants. for this article are available online. [ABSTRACT FROM AUTHOR]

Abstract–We provide a flexible framework for selecting among a class of additive partial linear models that allows both linear and nonlinear additive components. In practice, it is challenging to determine which additive components should be excluded from the model while simultaneously determining whether nonzero additive components should be represented as linear or nonlinear components in the final model. In this article, we propose a Bayesian model selection method that is facilitated by a carefully specified class of models, including the choice of a prior distribution and the nonparametric model used for the nonlinear additive components. We employ a series of latent variables that determine the effect of each variable among the three possibilities (no effect, linear effect, and nonlinear effect) and that simultaneously determine the knots of each spline for a suitable penalization of smooth functions. The use of a pseudo-prior distribution along with a collapsing scheme enables us to deploy well-behaved Markov chain Monte Carlo samplers, both for model selection and for fitting the preferred model. Our method and algorithm are deployed on a suite of numerical studies and are applied to a nutritional epidemiology study. The numerical results show that the proposed methodology outperforms previously available methods in terms of effective sample sizes of the Markov chain samplers and the overall misclassification rates. [ABSTRACT FROM AUTHOR]

There has been considerable recent interest in Bayesian modeling of high-dimensional networks via latent space approaches. When the number of nodes increases, estimation based on Markov chain Monte Carlo can be extremely slow and show poor mixing, thereby motivating research on alternative algorithms that scale well in high-dimensional settings. In this article, we focus on the latent factor model, a widely used approach for latent space modeling of network data. We develop scalable algorithms to conduct approximate Bayesian inference via stochastic optimization. Leveraging sparse representations of network data, the proposed algorithms show massive computational and storage benefits, and allow to conduct inference in settings with thousands of nodes. An R package with an efficient c++ implementation of the proposed algorithms is provided. [ABSTRACT FROM AUTHOR]

Mixture transition distribution (MTD) time series models build high-order dependence through a weighted combination of first-order transition densities for each one of a specified number of lags. We present a framework to construct stationary MTD models that extend beyond linear, Gaussian dynamics. We study conditions for first-order strict stationarity which allow for different constructions with either continuous or discrete families for the first-order transition densities given a prespecified family for the marginal density, and with general forms for the resulting conditional expectations. Inference and prediction are developed under the Bayesian framework with particular emphasis on flexible, structured priors for the mixture weights. Model properties are investigated both analytically and through synthetic data examples. Finally, Poisson and Lomax examples are illustrated through real data applications. Supplementary files for this article are available online. [ABSTRACT FROM AUTHOR]