Search

Phylogenetic and discrete-trait evolutionary inference depend heavily on an appropriate characterization of the underlying character substitution process. In this paper, we present random-effects substitution models that extend common continuous-time Markov chain models into a richer class of processes capable of capturing a wider variety of substitution dynamics. As these random-effects substitution models often require many more parameters than their usual counterparts, inference can be both statistically and computationally challenging. Thus, we also propose an efficient approach to compute an approximation to the gradient of the data likelihood with respect to all unknown substitution model parameters. We demonstrate that this approximate gradient enables scaling of sampling-based inference, namely Bayesian inference via Hamiltonian Monte Carlo, under random-effects substitution models across large trees and state-spaces. Applied to a dataset of 583 SARS-CoV-2 sequences, an HKY model with random-effects shows strong signals of nonreversibility in the substitution process, and posterior predictive model checks clearly show that it is a more adequate model than a reversible model. When analyzing the pattern of phylogeographic spread of 1441 influenza A virus (H3N2) sequences between 14 regions, a random-effects phylogeographic substitution model infers that air travel volume adequately predicts almost all dispersal rates. A random-effects state-dependent substitution model reveals no evidence for an effect of arboreality on the swimming mode in the tree frog subfamily Hylinae. Simulations reveal that random-effects substitution models can accommodate both negligible and radical departures from the underlying base substitution model. We show that our gradient-based inference approach is over an order of magnitude more time efficient than conventional approaches.

This article presents a Poisson common factor model with an overdispersion factor to predict some multiple populations' mortality rates. We use Bayesian data analysis and an extension of the Hamiltonian Monte Carlo sampler to compute the estimation of the model parameters and mortality rates prediction. We apply the proposed model to the real mortality data of some European countries. Using some model selection measures, we compare the proposed model with a log-linear model and those introduced in Antonio, Bardoutsos, and Ouburg (2015, Bayesian Poisson log-bilinear models for mortality projections with multiple populations. European Actuarial Journal 5 (2): 245–281) and Wong, Forster, and Smith (2018, Bayesian mortality forecasting with overdispersion. Insurance: Mathematics and Economics 83: 206–221). [ABSTRACT FROM AUTHOR]

This article advocates proximal Markov chain Monte Carlo (ProxMCMC) as a flexible and general Bayesian inference framework for constrained or regularized estimation. Originally introduced in the Bayesian imaging literature, ProxMCMC employs the Moreau-Yosida envelope for a smooth approximation of the total-variation regularization term, fixes variance and regularization strength parameters as constants, and uses the Langevin algorithm for the posterior sampling. We extend ProxMCMC to be fully Bayesian by providing data-adaptive estimation of all parameters including the regularization strength parameter. More powerful sampling algorithms such as Hamiltonian Monte Carlo are employed to scale ProxMCMC to high-dimensional problems. Analogous to the proximal algorithms in optimization, ProxMCMC offers a versatile and modularized procedure for conducting statistical inference on constrained and regularized problems. The power of ProxMCMC is illustrated on various statistical estimation and machine learning tasks, the inference of which is traditionally considered difficult from both frequentist and Bayesian perspectives. [ABSTRACT FROM AUTHOR]

The efficiency of Hamiltonian Monte Carlo (HMC) can suffer when sampling a distribution with a wide range of length scales, because the small step sizes needed for stability in high-curvature regions are inefficient elsewhere. To address this we present a delayed rejection (DR) variant: if an initial HMC trajectory is rejected, we make one or more subsequent proposals each using a step size geometrically smaller than the last. To reduce the cost of DR approaches, we extend the standard delayed rejection to a probabilistic framework wherein we do not make multiple proposals at every rejection, but allow the probability of a retry to depend on the probability of accepting the previous proposal. We test the scheme in several sampling tasks, including statistical applications and multiscale model distributions such as Neal's funnel. Delayed rejection enables sampling multiscale distributions for which standard approaches such as HMC fail to explore the tails, and improves performance five-fold over optimally-tuned HMC as measured by ef- fective sample size per gradient evaluation. Even for simpler distributions, delayed rejection provides increased robustness to step size misspecification. [ABSTRACT FROM AUTHOR]

The stochastic volatility in mean (SVM) model proposed by Koopman and Uspensky (J Appl Econ 17:667–689, 2002) is revisited. This paper has two goals. The first is to offer a methodology that requires less computational time in simulations and estimates compared with others proposed in the literature as in Abanto-Valle et al. (Q Rev Econ Financ 80:272–286, 2021) and others. To achieve the first goal, we propose to approximate the likelihood function of the model applying Hidden Markov Models machinery to make possible Bayesian inference in real-time. We sample from the posterior distribution of parameters with a multivariate Normal distribution with mean and variance given by the posterior mode and the inverse of the Hessian matrix evaluated at this posterior mode using importance sampling. Further, the frequentist properties of estimators are analyzed conducting a simulation study. The second goal is to provide empirical evidence estimating the SVM model using daily data for five Latin American stock markets, USA, England, Japan and China. The results indicate that volatility negatively impacts returns, suggesting that the volatility feedback effect is stronger than the effect related to the expected volatility. This result is similar to the findings of Koopman and Uspensky (J Appl Econ 17:667–689, 2002), where the respective coefficient is negative but non statistically significant. However, in our case, all countries (except Peru and China) presents negative and statistically significant effects. Our results are similar to those found using Hamiltonian Monte Carlo (HMC) and Riemannian HMC methods based on Abanto-Valle et al. (Q Rev Econ Financ 80:272–286, 2021). [ABSTRACT FROM AUTHOR]

The past few decades have seen considerable interest in modeling time series of counts, with applications in many domains. Classical and Bayesian modeling have primarily focused on conditional Poisson sampling distributions at each time. There is very little research on modeling time series involving Zero-Modified (i.e., Zero Deflated or Inflated) distributions. This paper aims to fill this gap and develop models for count time series involving Zero-Modified distributions, which belong to the Power Series family and are suitable for time series exhibiting both zero-inflation and zero-deflation. A full Bayesian approach via the Hamiltonian Monte Carlo (HMC) technique enables accurate modeling and inference. The paper illustrates our approach using time series on the number of deaths from the influenza virus in the city of São Paulo, Brazil. [ABSTRACT FROM AUTHOR]

Molecular clock models undergird modern methods of divergence-time estimation. Local clock models propose that the rate of molecular evolution is constant within phylogenetic subtrees. Current local clock inference procedures exhibit one or more weaknesses, namely they achieve limited scalability to trees with large numbers of taxa, impose model misspecification, or require a priori knowledge of the existence and location of clocks. To overcome these challenges, we present an autocorrelated, Bayesian model of heritable clock rate evolution that leverages heavy-tailed priors with mean zero to shrink increments of change between branch-specific clocks. We further develop an efficient Hamiltonian Monte Carlo sampler that exploits closed form gradient computations to scale our model to large trees. Inference under our shrinkage clock exhibits a speed-up compared to the popular random local clock when estimating branch-specific clock rates on a variety of simulated datasets. This speed-up increases with the size of the problem. We further show our shrinkage clock recovers known local clocks within a rodent and mammalian phylogeny. Finally, in a problem that once appeared computationally impractical, we investigate the heritable clock structure of various surface glycoproteins of influenza A virus in the absence of prior knowledge about clock placement. We implement our shrinkage clock and make it publicly available in the BEAST software package.

Divergence time estimation is crucial to provide temporal signals for dating biologically important events from species divergence to viral transmissions in space and time. With the advent of high-throughput sequencing, recent Bayesian phylogenetic studies have analyzed hundreds to thousands of sequences. Such large-scale analyses challenge divergence time reconstruction by requiring inference on highly correlated internal node heights that often become computationally infeasible. To overcome this limitation, we explore a ratio transformation that maps the original $N-1$ internal node heights into a space of one height parameter and $N-2$ ratio parameters. To make the analyses scalable, we develop a collection of linear-time algorithms to compute the gradient and Jacobian-associated terms of the log-likelihood with respect to these ratios. We then apply Hamiltonian Monte Carlo sampling with the ratio transform in a Bayesian framework to learn the divergence times in 4 pathogenic viruses (West Nile virus, rabies virus, Lassa virus, and Ebola virus) and the coralline red algae. Our method both resolves a mixing issue in the West Nile virus example and improves inference efficiency by at least 5-fold for the Lassa and rabies virus examples as well as for the algae example. Our method now also makes it computationally feasible to incorporate mixed-effects molecular clock models for the Ebola virus example, confirms the findings from the original study, and reveals clearer multimodal distributions of the divergence times of some clades of interest.

We present the new R package instrument to perform Bayesian estimation of person explanatory multidimensional item response theory. The package implements an exploratory multidimensional item response theory model and a higher-order multidimensional item response theory model, a type of confirmatory multidimensional item response theory. Explanation of person parameters is accomplished by fixed and random effect linear regression models. Estimation is carried out using Hamiltonian Monte Carlo in Stan. In this article, we provide a detailed description of the models; we use the instrument package to demonstrate fitting explanatory item response models with fixed and random effects (i.e., mixed modeling) of person parameters in R; and, we perform a simulation study to evaluate the performance of our implementation of the models. [ABSTRACT FROM AUTHOR]

There has been substantial interest in developing Markov chain Monte Carlo algorithms based on piecewise deterministic Markov processes. However, existing algorithms can only be used if the target distribution of interest is differentiable everywhere. The key to adapting these algorithms so that they can sample from densities with discontinuities is to define appropriate dynamics for the process when it hits a discontinuity. We present a simple condition for the transition of the process at a discontinuity which can be used to extend any existing sampler for smooth densities, and give specific choices for this transition which work with popular algorithms such as the bouncy particle sampler, the coordinate sampler, and the zigzag process. Our theoretical results extend and make rigorous arguments that have been presented previously, for instance constructing samplers for continuous densities restricted to a bounded domain, and we present a version of the zigzag process that can work in such a scenario. Our novel approach to deriving the invariant distribution of a piecewise deterministic Markov process with boundaries may be of independent interest. [ABSTRACT FROM AUTHOR]

Analysis of brain connectivity is important for understanding how information is processed by the brain. We propose a novel Bayesian vector autoregression hierarchical model for analysing brain connectivity within resting-state functional magnetic resonance imaging, and apply it to simulated data and a real data set with subjects in different groups. Our approach models functional and effective connectivity simultaneously and allows for both group- and single-subject inference. We combine analytical marginalization with Hamiltonian Monte Carlo to obtain highly efficient posterior sampling. We show that our model gives similar inference for effective connectivity compared to models with a common covariance matrix to all subjects, but more accurate inference for functional connectivity between regions compared to models with more restrictive covariance structures. A Stan implementation of our model is available on GitHub. [ABSTRACT FROM AUTHOR]

AbstractThis work introduces a novel and efficient Bayesian federated learning algorithm, namely, the Federated Averaging stochastic Hamiltonian Monte Carlo (FA-HMC), for parameter estimation and uncertainty quantification. We establish rigorous convergence guarantees of FA-HMC on non-iid distributed datasets, under the strong convexity and Hessian smoothness assumptions. Our analysis investigates the effects of parameter space dimension, noise on gradients and momentum, and the frequency of communication (between the central node and local nodes) on the convergence and communication costs of FA-HMC. Beyond that, we establish the tightness of our analysis by showing that the convergence rate cannot be improved even for continuous FA-HMC process. Moreover, extensive empirical studies demonstrate that FA-HMC outperforms the existing Federated Averaging-Langevin Monte Carlo (FA-LD) algorithm. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

Ordinal response is commonly found in medicine, biology, and other fields. In many situations, the predictors for this ordinal response are compositional, which means that the sum of predictors for each sample is fixed. Examples of compositional data include the relative abundance of species in microbiome data and the relative frequency of nutrition concentrations. Moreover, the predictors that are strongly correlated tend to have similar influence on the response outcome. Conventional cumulative logistic regression models for ordinal responses ignore the fixed-sum constraint on predictors and their associated interrelationships, and thus are not appropriate for analyzing compositional predictors. To solve this problem, we proposed Bayesian Compositional Models for Ordinal Response to analyze the relationship between compositional data and an ordinal response with a structured regularized horseshoe prior for the compositional coefficients and a soft sum-to-zero restriction on coefficients through the prior distribution. The method was implemented with R package rstan using efficient Hamiltonian Monte Carlo algorithm. We performed simulations to compare the proposed approach and existing methods for ordinal responses. Results revealed that our proposed method outperformed the existing methods in terms of parameter estimation and prediction. We also applied the proposed method to a microbiome study HMP2Data, to find microorganisms linked to ordinal inflammatory bowel disease levels. To make this work reproducible, the code and data used in this paper are available at https://github.com/Li-Zhang28/BCO. [ABSTRACT FROM AUTHOR]

Volatility modeling is crucial in finance, especially when dealing with intraday transaction‐level asset returns. The irregular and high‐frequency nature of the data presents unique challenges. While stochastic volatility (SV) models are widely used for understanding patterns in volatility of daily stock returns which constitute regularly spaced time series, new classes of models must be introduced for analyzing volatility in irregularly spaced intraday data. Specifically these models must accommodate the random gaps between successive transactional events. By modeling the gaps using autoregressive conditional duration (ACD) models, we describe a hierarchical irregular SV autoregressive conditional duration (IR‐SV‐ACD) model for estimating and forecasting intertransaction gaps and the volatility of log‐returns. We carry out the analysis in the Bayesian framework via the Hamiltonian Monte Carlo (HMC) algorithm with No‐U‐turn sampler (NUTS) in R using the cmdstanr package. The fits and forecasts are obtained using Monte Carlo averages based on the posterior samples. We illustrate this approach using simulation studies and real data analysis for intraday prices available at microseconds level of health stocks traded on the New York Stock Exchange (NYSE). The log‐returns and gaps are calculated for the stocks and are used for modeling. [ABSTRACT FROM AUTHOR]

The simultaneous estimation of multiple quantiles is a crucial statistical task that enables a thorough understanding of data distribution for robust analysis and decision-making. In this study, we adopt a Bayesian approach to tackle this critical task, employing the asymmetric Laplace distribution (ALD) as a flexible framework for quantile modeling. Our methodology implementation involves the Hamiltonian Monte Carlo (HMC) algorithm, building on the foundation laid in prior work, where the error term is assumed to follow an ALD. Capitalizing on the interplay between two distinct quantiles of this distribution, we endorse a straightforward and fully Bayesian method that adheres to the non-crossing property of quantiles. Illustrated through simulated scenarios, we showcase the effectiveness of our approach in quantile estimation, enhancing precision via the HMC algorithm. The proposed method proves versatile, finding application in finance, environmental science, healthcare, and manufacturing, and contributing to sustainable development goals by fostering innovation and enhancing decision-making in diverse fields. [ABSTRACT FROM AUTHOR]

This study proposes a method to infer the parameters of the constant elasticity of variance (CEV) model from the market values of stock after the extension from the asset process of the Merton model in the structural credit risk model to that of the CEV model. The state space model is used, which consists of an asset process (system equation) and the call option pricing a stock value (observation equation), for the inference. However, it is usually difficult to apply the Markov chain Monte Carlo (MCMC) method to estimate the parameters of the CEV model because the observation equation of the state space model has no analytical formula. Our method solves this parameter estimation problem by applying the MCMC combined with a finite difference method of partial differential equations, where the stock value obtained as a CEV option price is numerically solved. This study estimates the parameters from the real stock values of the US financial institutions as an empirical analysis. Furthermore, we analyze the default probability and measure the credit risk of bank portfolios. [ABSTRACT FROM AUTHOR]

Compositional data are vectors typically representing proportions of a whole, that is, those whose elements are strictly positive and subject to a unit-sum constraint. The increasing number of fields where this type of data arises makes the development of proper statistical tools an important issue. From a regression perspective, whenever the multivariate response is a compositional vector, a proper model that accounts for the unit-sum constraint is the well-established Dirichlet regression model. However, there are significant drawbacks mainly due to the limited flexibility of the Dirichlet distribution. The aim of this contribution is to introduce a new multivariate regression model for constrained responses, that is based on the extended flexible Dirichlet distribution (which is a structured mixture with Dirichlet distributed components). The new model is obtained by adopting a novel reparameterization which allows for, among other things, the presence of suitably designed cluster-specific regression patterns. It is shown to provide considerably greater flexibility and better performance than the standard Dirichlet regression model. In particular, from theoretical analysis, intensive simulation studies in many challenging scenarios, as well as from a real data application, it emerges that the new regression model can handle several issues affecting the Dirichlet regression, such as the presence of outliers, latent groups, multi-modality, and positive correlations. [ABSTRACT FROM AUTHOR]

Proximal Markov Chain Monte Carlo is a novel construct that lies at the intersection of Bayesian computation and convex optimization, which helped popularize the use of nondifferentiable priors in Bayesian statistics. Existing formulations of proximal MCMC, however, require hyperparameters and regularization parameters to be prespecified. In this work, we extend the paradigm of proximal MCMC through introducing a novel new class of nondifferentiable priors called epigraph priors. As a proof of concept, we place trend filtering, which was originally a nonparametric regression problem, in a parametric setting to provide a posterior median fit along with credible intervals as measures of uncertainty. The key idea is to replace the nonsmooth term in the posterior density with its Moreau-Yosida envelope, which enables the application of the gradient-based MCMC sampler Hamiltonian Monte Carlo. The proposed method identifies the appropriate amount of smoothing in a data-driven way, thereby automating regularization parameter selection. Compared with conventional proximal MCMC methods, our method is mostly tuning free, achieving simultaneous calibration of the mean, scale and regularization parameters in a fully Bayesian framework.

Using Bayesian methods to apply computational models of cognitive processes, or Bayesian cognitive modeling, is an important new trend in psychological research. The rise of Bayesian cognitive modeling has been accelerated by the introduction of software that efficiently automates the Markov chain Monte Carlo sampling used for Bayesian model fitting-including the popular Stan and PyMC packages, which automate the dynamic Hamiltonian Monte Carlo and No-U-Turn Sampler (HMC/NUTS) algorithms that we spotlight here. Unfortunately, Bayesian cognitive models can struggle to pass the growing number of diagnostic checks required of Bayesian models. If any failures are left undetected, inferences about cognition based on the model's output may be biased or incorrect. As such, Bayesian cognitive models almost always require troubleshooting before being used for inference. Here, we present a deep treatment of the diagnostic checks and procedures that are critical for effective troubleshooting, but are often left underspecified by tutorial papers. After a conceptual introduction to Bayesian cognitive modeling and HMC/NUTS sampling, we outline the diagnostic metrics, procedures, and plots necessary to detect problems in model output with an emphasis on how these requirements have recently been changed and extended. Throughout, we explain how uncovering the exact nature of the problem is often the key to identifying solutions. We also demonstrate the troubleshooting process for an example hierarchical Bayesian model of reinforcement learning, including supplementary code. With this comprehensive guide to techniques for detecting, identifying, and overcoming problems in fitting Bayesian cognitive models, psychologists across subfields can more confidently build and use Bayesian cognitive models in their research. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Beta distributions are commonly used to model proportion valued response variables, often encountered in longitudinal studies. In this article, we develop semi-parametric Beta regression models for proportion valued responses, where the aggregate covariate effect is summarized and flexibly modeled, using a interpretable monotone time-varying single index transform of a linear combination of the potential covariates. We utilize the potential of single index models, which are effective dimension reduction tools and accommodate link function misspecification in generalized linear mixed models. Our Bayesian methodology incorporates the missing-at-random feature of the proportion response and utilize Hamiltonian Monte Carlo sampling to conduct inference. We explore finite-sample frequentist properties of our estimates and assess the robustness via detailed simulation studies. Finally, we illustrate our methodology via application to a motivating longitudinal dataset on obesity research recording proportion body fat. [ABSTRACT FROM AUTHOR]

AbstractIn this paper, we deal with a Bayesian inference method for estimating the parameters of the queueing system with short- or long-tailed distributions based on the No-U-Turn Sampler (NUTS), a recently developed Hamilton Monte Carlo (HMC). We assume inter-arrival and service times to be either the short-tailed distributions or the long-tailed distributions since they are a better fit for real-world data. We illustrate our assumption using a number of simulated data sets, generated from distributions covering a wide range of cases. Then we estimate the parameters using the Bayesian approach based on No-U-Turn Sampler. As a result of comparing the No-U-Turn Sampler with the Gibbs sampler, the most common MCMC algorithm, we demonstrate that the NUTS outperforms Gibbs sampler for estimating parameters, which is especially significant for long-tailed distribution. We also investigate the influence of the size of observation data and the prior distributions on estimating these parameters. [ABSTRACT FROM AUTHOR]

The choice of a prior distribution is a key aspect of the Bayesian method. However, in many cases, such as the family of power links, this is not trivial. In this article, we introduce a penalized complexity prior (PC prior) of the skewness parameter for this family, which is useful for dealing with imbalanced data. We derive a general expression for this density and show its usefulness for some particular cases such as the power logit and the power probit links. A simulation study and a real data application are used to assess the efficiency of the introduced densities in comparison with the Gaussian and uniform priors. Results show improvement in point and credible interval estimation for the considered models when using the PC prior in comparison to other well‐known standard priors. [ABSTRACT FROM AUTHOR]

Recent advances in Bayesian phylogenetics offer substantial computational savings to accommodate increased genomic sampling that challenges traditional inference methods. In this review, we begin with a brief summary of the Bayesian phylogenetic framework, and then conceptualize a variety of methods to improve posterior approximations via Markov chain Monte Carlo (MCMC) sampling. Specifically, we discuss methods to improve the speed of likelihood calculations, reduce MCMC burn-in, and generate better MCMC proposals. We apply several of these techniques to study the evolution of HIV virulence along a 1536-tip phylogeny and estimate the internal node heights of a 1000-tip SARS-CoV-2 phylogenetic tree in order to illustrate the speed-up of such analyses using current state-of-the-art approaches. We conclude our review with a discussion of promising alternatives to MCMC that approximate the phylogenetic posterior. This article is part of a discussion meeting issue 'Genomic population structures of microbial pathogens'.

This article studies long-term, short-term volatility and co-volatility in stock markets by introducing modelling strategies to the multivariate data analysis that deal with serially correlated innovations and cross-section dependence. In particular, it presents an innovative mixed-effects model through a GARCH process, allowing for heterogeneity effects and time-series dynamics. We propose a non-parametric regression model of the penalized low-rank smoothing spline to present time trends into the variance and covariance equations. The strategy provides flexible modelling of the low-frequency volatility and co-volatility in equity markets. The decomposed low-frequency matrix was modelled using the modified Cholesky factorization. The Hamiltonian Monte Carlo technique is implemented as a Bayesian computing process for estimating parameters and latent factors. The advantage of our modelling strategy in empirical studies is highlighted by examining the effect of latent financial factors on a panel across 10 equities over 110 weekly series. The model can differentiate non-parametrically dynamic patterns of high and low frequencies of variance–covariance structural equations and incorporate economic features to predict variabilities in stock markets regarding time-series evidence. [ABSTRACT FROM AUTHOR]

Traffic intensity is one of the most critical parameters of single-server Markovian queues. This paper deals with the Bayesian inference for the M/M/1 queue by sampling from the posterior distribution. The No-U-Turn Sampler (NUTS) is a recently developed Markov Chain Monte Carlo (MCMC) algorithm, which is proposed to compute the traffic intensity by observing the number of customers in the system at the departure epoch. Numerical results show that the NUTS outperforms the other algorithms in the literature. [ABSTRACT FROM AUTHOR]

The continuous-time Markov chain (CTMC) is the mathematical workhorse of evolutionary biology. Learning CTMC model parameters using modern, gradientbased methods requires the derivative of the matrix exponential evaluated at the CTMC's infinitesimal generator (rate) matrix. Motivated by the derivative's extreme computational complexity as a function of state space cardinality, recent work demonstrates the surprising effectiveness of a naive, first-order approximation for a host of problems in computational biology. In response to this empirical success, we obtain rigorous deterministic and probabilistic bounds for the error accrued by the naive approximation and establish a "blessing of dimensionality" result that is universal for a large class of rate matrices with random entries. Finally, we apply the first-order approximation within surrogate-trajectory Hamiltonian Monte Carlo for the analysis of the early spread of Severe acute respiratory syndrome coronavirus 2 (SARSCoV- 2) across 44 geographic regions that comprise a state space of unprecedented dimensionality for unstructured (flexible) CTMC models within evolutionary biology. [ABSTRACT FROM AUTHOR]

Bayesian physics-informed neural networks (B-PINNs) have emerged as an efficient tool for uncertainty quantification in partial differential equations (PDEs). However, their applicability has been limited to accounting for noisy data. They fail to effectively address the uncertainty arising from the randomness of physical parameters in stochastic PDEs (SPDEs). To this end, we propose a novel Bayesian deep learning framework designed for uncertainty quantification in SPDE problems. We model the solution of an SPDE using a Bayesian neural network (BNN), where the posterior distribution of the parameters is constructed by Bayes' rule. We incorporate the governing physical laws into the likelihood distribution through automatic differentiation and estimate the likelihood using a Gaussian mixture model. To effectively learn the posterior distribution of BNN parameters, we employ a Hamiltonian Monte Carlo method, an efficient method for high-dimensional sampling. We propose a parallel algorithm to mitigate the high computational cost associated with the Hamiltonian Monte Carlo, thereby enhancing its efficiency. We provide numerical examples for both forward and inverse SPDE problems to demonstrate the effectiveness of our proposed method for uncertainty quantification in SPDEs. Notably, it is demonstrated that the computational cost is almost unaffected by the number of BNN parameters and the random dimension of the problem, underscoring its computational efficiency. [ABSTRACT FROM AUTHOR]

The COVID-19 pandemic has added to the challenge of decarbonizing the transportation sector, as shared modes were perceived as more dangerous during the health emergency. If these behaviors persist, drawing riders to more sustainable modes may be more difficult. This study investigates measures how crowding multipliers in New York City for the subway, ridehailing, and microtransit changed during and after the pandemic. We used Bayesian techniques to estimate two mixed logit models based on stated preference data. Results show that post-pandemic crowding multipliers are either similar or lower than during the pandemic, depending on the transportation mode and masking compliance. Additionally, vaccination requirements did not significantly affect respondents' choices, but respondents were willing to pay to reduce their transportation mode's carbon footprint. The study suggests that commuters' aversion to crowding will gradually decrease, but whether crowding multipliers will return to pre-pandemic levels or a post-pandemic "new normal" remains uncertain. • COVID-19 makes decarbonizing transportation harder, creating preference for private vehicles. • We obtained crowding multipliers for ridehailing, microtransit and subway during and post-pandemic using stated preference data from NYC. • Post-pandemic multipliers similar or lower than pandemic ones, depending on mode and masking. • More masking slightly reduces crowding multipliers during the pandemic. • Vaccination requirements do not affect decisions; willingness to pay to cut carbon emissions found. [ABSTRACT FROM AUTHOR]

We consider a Markov chain Monte Carlo method, known as Hamiltonian Monte Carlo (HMC), on compact manifolds in Euclidean space. It utilizes Hamiltonian dynamics to generate samples approximating a target distribution in high dimensions efficiently. The efficiency of HMC is characterized by its convergence property, called geometric ergodicity. This property is important to generate low-correlated samples. It also plays a crucial role in establishing the error estimate for the quadrature of bounded functions by HMC sampling, referred to as the Hoeffding-type inequality. While the geometric ergodicity has been proved for HMC on Euclidean space, it has not been established on manifolds. In this paper, we prove the geometric ergodicity for HMC on compact manifolds. As an example to confirm the efficiency of the proposed HMC method, we consider a sampling problem associated with the N-vortex problem on the unit sphere, which is a statistical model of two-dimensional turbulence. We apply HMC to approximate the statistical quantities with respect to the invariant measure of the N-vortex problem, called the Gibbs measure. We observe the organization of large vortex structures as seen in two-dimensional turbulence. [ABSTRACT FROM AUTHOR]

We develop a new family of marked point processes by focusing the characteristic properties of marked Hawkes processes exclusively on the space of marks, allowing a separate model specification for the occurrence times. We develop a Bayesian framework for their inference and prediction that can naturally accommodate covariate information to drive cross-excitations, offering broad flexibility for real-world applications. The framework is applied to in-game event sequences from association football, resulting in inferences about previously unquantified characteristics of game dynamics, extraction of event-specific team abilities and predictions for event occurrences, such as goals or fouls in a specified interval of time. [ABSTRACT FROM AUTHOR]

Among Markov chain Monte Carlo algorithms, Hamiltonian Monte Carlo (HMC) is often the algorithm of choice for complex, high-dimensional target distributions; however, its efficiency is notoriously sensitive to the choice of the integration-time tuning parameter. When integrating both forward and backward in time using the same leapfrog integration step as HMC, the set of apogees, local maxima in the potential along a path, is the same whatever point (position and momentum) along the path is chosen to initialize the integration. We present the Apogee to Apogee Path Sampler (AAPS), which uses this invariance to create a simple yet generic methodology for constructing a path, proposing a point from it and accepting or rejecting that proposal so as to target the intended distribution. We demonstrate empirically that AAPS has a similar efficiency to HMC but is much more robust to the setting of its equivalent tuning parameter, the number of apogees that the path crosses. for this article are available online. [ABSTRACT FROM AUTHOR]

We investigate solution methods for large-scale inverse problems governed by partial differential equations (PDEs) via Bayesian inference. The Bayesian framework provides a statistical setting to infer uncertain parameters from noisy measurements. To quantify posterior uncertainty, we adopt Markov Chain Monte Carlo (MCMC) approaches for generating samples. To increase the efficiency of these approaches in high-dimension, we make use of local information about gradient and Hessian of the target potential, also via Hamiltonian Monte Carlo (HMC). Our target application is inferring the field of soil permeability processing observations of pore pressure, using a nonlinear PDE poromechanics model for predicting pressure from permeability. We compare the performance of different sampling approaches in this and other settings. We also investigate the effect of dimensionality and non-gaussianity of distributions on the performance of different sampling methods. [ABSTRACT FROM AUTHOR]

In this article, we consider the preconditioned Hamiltonian Monte Carlo (pHMC) algorithm defined directly on an infinite-dimensional Hilbert space. In this context, and under a condition reminiscent of strong log-concavity of the target measure, we prove convergence bounds for adjusted pHMC in the standard 1-Wasserstein distance. The arguments rely on a synchronous coupling of two copies of pHMC, which is controlled by adapting elements from Bou-Rabee, Eberle and Zimmer (2020). [ABSTRACT FROM AUTHOR]

Relaxed random walk (RRW) models of trait evolution introduce branch-specific rate multipliers to modulate the variance of a standard Brownian diffusion process along a phylogeny and more accurately model overdispersed biological data. Increased taxonomic sampling challenges inference under RRWs as the number of unknown parameters grows with the number of taxa. To solve this problem, we present a scalable method to efficiently fit RRWs and infer this branch-specific variation in a Bayesian framework. We develop a Hamiltonian Monte Carlo (HMC) sampler to approximate the high-dimensional, correlated posterior that exploits a closed-form evaluation of the gradient of the trait data log-likelihood with respect to all branch-rate multipliers simultaneously. Our gradient calculation achieves computational complexity that scales only linearly with the number of taxa under study. We compare the efficiency of our HMC sampler to the previously standard univariable Metropolis-Hastings approach while studying the spatial emergence of the West Nile virus in North America in the early 2000s. Our method achieves at least a 6-fold speed increase over the univariable approach. Additionally, we demonstrate the scalability of our method by applying the RRW to study the correlation between five mammalian life history traits in a phylogenetic tree with $3650$ tips.[Bayesian inference; BEAST; Hamiltonian Monte Carlo; life history; phylodynamics, relaxed random walk.].

Big Bayes is the computationally intensive co-application of big data and large, expressive Bayesian models for the analysis of complex phenomena in scientific inference and statistical learning. Standing as an example, Bayesian multidimensional scaling (MDS) can help scientists learn viral trajectories through space-time, but its computational burden prevents its wider use. Crucial MDS model calculations scale quadratically in the number of observations. We partially mitigate this limitation through massive parallelization using multi-core central processing units, instruction-level vectorization and graphics processing units (GPUs). Fitting the MDS model using Hamiltonian Monte Carlo, GPUs can deliver more than 100-fold speedups over serial calculations and thus extend Bayesian MDS to a big data setting. To illustrate, we employ Bayesian MDS to infer the rate at which different seasonal influenza virus subtypes use worldwide air traffic to spread around the globe. We examine 5392 viral sequences and their associated 14 million pairwise distances arising from the number of commercial airline seats per year between viral sampling locations. To adjust for shared evolutionary history of the viruses, we implement a phylogenetic extension to the MDS model and learn that subtype H3N2 spreads most effectively, consistent with its epidemic success relative to other seasonal influenza subtypes. Finally, we provide MassiveMDS, an open-source, stand-alone C++ library and rudimentary R package, and discuss program design and high-level implementation with an emphasis on important aspects of computing architecture that become relevant at scale.

The simultaneous estimation of multiple quantiles is a crucial statistical task that enables a thorough understanding of data distribution for robust analysis and decision-making. In this study, we adopt a Bayesian approach to tackle this critical task, employing the asymmetric Laplace distribution (ALD) as a flexible framework for quantile modeling. Our methodology implementation involves the Hamiltonian Monte Carlo (HMC) algorithm, building on the foundation laid in prior work, where the error term is assumed to follow an ALD. Capitalizing on the interplay between two distinct quantiles of this distribution, we endorse a straightforward and fully Bayesian method that adheres to the non-crossing property of quantiles. Illustrated through simulated scenarios, we showcase the effectiveness of our approach in quantile estimation, enhancing precision via the HMC algorithm. The proposed method proves versatile, finding application in finance, environmental science, healthcare, and manufacturing, and contributing to sustainable development goals by fostering innovation and enhancing decision-making in diverse fields.

Due to our limited knowledge about complex environmental systems, our predictions of their behavior under different scenarios or decision alternatives are subject to considerable uncertainty. As this uncertainty can often be relevant for societal decisions, the consideration, quantification and communication of it is very important. Due to internal stochasticity, often poorly known influence factors, and only partly known mechanisms, in many cases, a stochastic model is needed to get an adequate description of uncertainty. As this implies the need to infer constant parameters, as well as the time-course of stochastic model states, a very high-dimensional inference problem for model calibration has to be solved. This is very challenging from a methodological and a numerical perspective. To illustrate aspects of this problem and show options to successfully tackle it, we compare three numerical approaches: Hamiltonian Monte Carlo, Particle Markov Chain Monte Carlo, and Conditional Ornstein-Uhlenbeck Sampling. As a case study, we select the analysis of hydrological data with a stochastic hydrological model. We conclude that the performance of the investigated techniques is comparable for the analyzed system, and that also generality and practical considerations may be taken into account to guide the choice of which technique is more appropriate for a particular application. [ABSTRACT FROM AUTHOR]

We propose a Bayesian elastic net that uses empirical likelihood and develop an efficient tuning of Hamiltonian Monte Carlo for posterior sampling. The proposed model relaxes the assumptions on the identity of the error distribution, performs well when the variables are highly correlated, and enables more straightforward inference by providing posterior distributions of the regression coefficients. The Hamiltonian Monte Carlo method implemented in Bayesian empirical likelihood overcomes the challenges that the posterior distribution lacks a closed analytic form and its domain is nonconvex. We develop the leapfrog parameter tuning algorithm for Bayesian empirical likelihood. We also show that the posterior distributions of the regression coefficients are asymptotically normal. Simulation studies and real data analysis demonstrate the advantages of the proposed method in prediction accuracy. [ABSTRACT FROM AUTHOR]

Standard Markov chain Monte Carlo methods struggle to explore distributions that concentrate in the neighbourhood of low-dimensional submanifolds. This pathology naturally occurs in Bayesian inference settings when there is a high signal-to-noise ratio in the observational data but the model is inherently over-parametrised or nonidentifiable. In this paper, we propose a strategy that transforms the original sampling problem into the task of exploring a distribution supported on a manifold embedded in a higher-dimensional space; in contrast to the original posterior this lifted distribution remains diffuse in the limit of vanishing observation noise. We employ a constrained Hamiltonian Monte Carlo method, which exploits the geometry of this lifted distribution, to perform efficient approximate inference. We demonstrate in numerical experiments that, contrarily to competing approaches, the sampling efficiency of our proposed methodology does not degenerate as the target distribution to be explored concentrates near low-dimensional submanifolds. Python code reproducing the results is available at https://doi.org/10.5281/zenodo.6551654. [ABSTRACT FROM AUTHOR]

We establish L 픮 convergence for Hamiltonian Monte Carlo (HMC) algorithms. More specifically, under mild conditions for the associated Hamiltonian motion, we show that the outputs of the algorithms converge (strongly for 2 ≤ 픮 < ∞ and weakly for 1 < 픮 < 2 ) to the desired target distribution. In addition, we establish a general convergence rate for an L 픮 convergence given a convergence rate at a specific q * , and apply this result to conclude geometric convergence in the Euclidean space for HMC with uniformly strongly logarithmic concave target and auxiliary distributions. We also present the results of experiments to illustrate convergence in L 픮 . [ABSTRACT FROM AUTHOR]

This article reviews the Bayesian inference with the Monte Carlo Markov Chain (MCMC) and the Hamiltonian Monte Carlo (HMC) samplers as a competitor of the classical likelihood statistical inference for pharmacometricians. The MCMC and the HMC samplers have greatly contributed to realization of the Bayesian methods with minimal requirement of mathematical theory. They do not require any closed form of the posterior density nor linear approximation of complex nonlinear models in high dimension even with non-conjugate priors. The HMC even weakens the dependency of the chain and improves computational efficiency. Pharmacometrics is one of great beneficiaries since they use complex multivariate multilevel nonlinear mixed effects models based on the restricted maximum likelihood estimation. Comprehension of the Bayesian approach will help pharmacometricians to access the data analysis more conveniently. [ABSTRACT FROM AUTHOR]

Various Markov chain Monte Carlo (MCMC) methods are studied to improve upon random walk Metropolis sampling, for simulation from complex distributions. Examples include Metropolis-adjusted Langevin algorithms, Hamiltonian Monte Carlo, and other algorithms related to underdamped Langevin dynamics. We propose a broad class of irreversible sampling algorithms, called Hamiltonian-assisted Metropolis sampling (HAMS), and develop two specific algorithms with appropriate tuning and preconditioning strategies. Our HAMS algorithms are designed to simultaneously achieve two distinctive properties, while using an augmented target density with a momentum as an auxiliary variable. One is generalized detailed balance, which induces an irreversible exploration of the target. The other is a rejection-free property for a Gaussian target with a prespecified variance matrix. This property allows our preconditioned algorithms to perform satisfactorily with relatively large step sizes. Furthermore, we formulate a framework of generalized Metropolis–Hastings sampling, which not only highlights our construction of HAMS at a more abstract level, but also facilitates possible further development of irreversible MCMC algorithms. We present several numerical experiments, where the proposed algorithms consistently yield superior results among existing algorithms using the same preconditioning schemes. [ABSTRACT FROM AUTHOR]

We propose a Bayesian hidden Markov model for analyzing time series and sequential data where a special structure of the transition probability matrix is embedded to model explicit-duration semi-Markovian dynamics. Our formulation allows for the development of highly flexible and interpretable models that can integrate available prior information on state durations while keeping a moderate computational cost to perform efficient posterior inference. We show the benefits of choosing a Bayesian approach for HSMM estimation over its frequentist counterpart, in terms of model selection and out-of-sample forecasting, also highlighting the computational feasibility of our inference procedure whilst incurring negligible statistical error. The use of our methodology is illustrated in an application relevant to e-Health, where we investigate rest-activity rhythms using telemetric activity data collected via a wearable sensing device. This analysis considers for the first time Bayesian model selection for the form of the explicit state dwell distribution. We further investigate the inclusion of a circadian covariate into the emission density and estimate this in a data-driven manner. [ABSTRACT FROM AUTHOR]