Search

In this paper we propose a novel approach to estimating and testing skewness in a stochastic volatility (SV) model. Our key idea is to replace a normal return error in the standard SV model with a split normal error. We show that this simple variation in the model brings about two large computational advantages. First, the SV can be simulated fast and efficiently using a one-block Gibbs sampling technique. Second, more importantly, this is the first to provide a marginal likelihood calculation method to formally test the skewness and SV in a Bayesian framework. We subsequently demonstrate the efficiency and reliability of our posterior sampling and model comparison methods through a simulation study. The simulation study results also show that neglecting skewness leads to inaccurate SV estimates and conditional expected returns. Our empirical applications to daily stock return data also show strong evidence of negative skewness.

Topic modeling is a significant branch of natural language processing and machine learning focused on inferring the generative process of text. Traditionally, algorithms for estimating topic models have relied on Bayesian inference and Gibbs sampling. This paper proposes a novel “acceptable set” framework for formulating topic modeling problems inspired by ideas from discrete component analysis and data-driven robust optimization. Our approach not only simplifies the design and inference of topic models, but also allows for extensions and generalizations that are challenging to integrate into traditional approaches. Different restrictions (e.g., sparsity) and assumptions (e.g., alternative generative processes) can be easily incorporated into our formulations through additional or modified constraints. Our formulations also naturally control a widely used metric of model quality, perplexity. We adapt state-of-the-art stochastic gradient methods to find good local optima for the optimization formulations. The algorithms are efficient, scaling to realistic problem sizes with runtimes comparable to existing methods. Through extensive computational experiments, we show that our methods have improved solution quality compared to baseline methods and reconstruct more reliably the underlying generative models. Our framework overcomes known vulnerabilities of traditional topic modeling algorithms: our methods are effective in low-data settings, register good out-of-sample performance, and perform well for a variety of initial assumptions on input parameter values.

Recently, listwise collaborative filtering (CF) algorithms are attracting increasing interest due to their efficiency and prediction quality. Different from rating-oriented (pointwise) CF, they recommend a preference ranking of items to each user without estimating the absolute value of the ratings. In practice, there are extensive side information about users and items in many corpora that can improve recommendation performance and specially help CF methods to address the cold-start problem. However, a model has not been proposed that incorporates side information to listwise CF. Therefore, in this work, a Bayesian graphical model, called Content-Aware Listwise Collaborative Filtering (CALCF), is developed which incorporates text information, represented as a bag of words, to listwise CF using topic models. We propose a Gibbs sampler with closed-form samples using data augmentation techniques to infer the latent variables. CALCF has been validated by comparison with previous listwise and pointwise algorithms on implicit (click, view, purchase) and explicit (numeric ratings) feedback data in both cold-start and warm-start scenarios. The results demonstrate that in most cases CALCF achieves better Normalized Discounted Cumulative Gain (NDCG) and Recall at top M recommendation compared with the previous models.

Community detection using statistical models is a promising research area in network analysis. Most existing statistical models for this task cannot be fitted well for various community structures. In this paper, we propose a model incorporating the latent link patterns for detecting effectively various community structures. It can automatically discover the number and the sizes of communities by grouping the vertices owning the same latent link pattern, which is meaningful and explainable. An inference approach based on collapsed Gibbs sampling is proposed to estimate the parameters of our model. Experiments on 13 real-world networks demonstrate our model outperforms four state-of-the-art approaches on most of the datasets and is competent to explore various community structures. In addition, our model can detect some hidden link patterns that offer extra information for network analysis.

An innovative model-based approach to coupling text clustering and topic modeling is introduced, in which the two tasks take advantage of each other. Specifically, the integration is enabled by a new generative model of text corpora. This explains topics, clusters and document content via a Bayesian generative process. In this process, documents include word vectors, to capture the (syntactic and semantic) regularities among words. Topics are multivariate Gaussian distributions on word vectors. Clusters are assigned corresponding topic distributions as their semantics. Content generation is ruled by text clusters and topics, which act as interacting latent factors. Documents are at first placed into respective clusters, then the semantics of these clusters is then repeatedly sampled to draw document topics, which are in turn sampled for word-vector generation. Under the proposed model, collapsed Gibbs sampling is derived mathematically and implemented algorithmically with parameter estimation for the simultaneous inference of text clusters and topics. A comparative assessment on real-world benchmark corpora demonstrates the effectiveness of this approach in clustering texts and uncovering their semantics. Intrinsic and extrinsic criteria are adopted to investigate its topic modeling performance, whose results are shown through a case study. Time efficiency and scalability are also studied.

Active learning is a machine learning paradigm allowing to decide which inputs to use for training. It is introduced to Genetic Programming (GP) essentially thanks to the dynamic data sampling, used to address some known issues such as the computational cost, the over-fitting problem and the imbalanced databases. The traditional dynamic sampling for GP gives to the algorithm a new sample periodically, often each generation, without considering the state of the evolution. In so doing, individuals do not have enough time to extract the hidden knowledge. An alternative approach is to use some information about the learning state to adapt the periodicity of the training data change. In this work, we propose an adaptive sampling strategy for classification tasks based on the state of solved fitness cases throughout learning. It is a flexible approach that could be applied with any dynamic sampling. We implemented some sampling algorithms extended with dynamic and adaptive controlling re-sampling frequency. We experimented them to solve the KDD intrusion detection and the Adult incomes prediction problems with GP. The experimental study demonstrates how the sampling frequency control preserves the power of dynamic sampling with possible improvements in learning time and quality. We also demonstrate that adaptive sampling can be an alternative to multi-level sampling. This work opens many new relevant extension paths.

Non-negative matrix factorization (NMF) is a knowledge discovery method that is used in many fields. Variational inference and Gibbs sampling methods for it are also well-known. However, the variational approximation error has not been clarified yet, because NMF is not statistically regular and the prior distribution used in variational Bayesian NMF (VBNMF) has zero or divergence points. In this paper, using algebraic geometrical methods, we theoretically analyze the difference in negative log evidence (a.k.a. free energy) between VBNMF and Bayesian NMF, i.e., the Kullback–Leibler divergence between the variational posterior and the true posterior. We derive an upper bound for the learning coefficient (a.k.a. the real log canonical threshold) in Bayesian NMF. By using the upper bound, we find a lower bound for the approximation error, asymptotically. The result quantitatively shows how well the VBNMF algorithm can approximate Bayesian NMF; the lower bound depends on the hyperparameters and the true non-negative rank. A numerical experiment demonstrates the theoretical result.

Regularised regression uses sparsity and variance to reduce the complexity and over-fitting of a regression model. The present paper introduces two novel regularised linear regression algorithms: Competitive Iterative Ridge Regression (CIRR) and Online Shrinkage via Limit of Gibbs Sampler (OSLOG) for fast and reliable prediction on “Big Data” without making distributional assumption on the data. We use the technique of competitive analysis to design them and show their strong theoretical guarantee. Furthermore, we compare their performance against some neoteric regularised regression methods such as Online Ridge Regression (ORR) and the Aggregating Algorithm for Regression (AAR). The comparison of the algorithms is done theoretically, focusing on the guarantee on the performance on cumulative loss, and empirically to show the advantages of CIRR and OSLOG.

The set of all frequent patterns that are extracted from a single network can be huge. A technique recently proposed for obtaining a compact, informative and useful set of patterns is output sampling, where a small set of frequent patterns is randomly chosen. However, existing output sampling algorithms work only in the transactional setting, where the database consists of a collection of relatively small graphs. In this paper, first we extend the output sampling framework to the single network setting where the database is a large single graph, counting supports of patterns is more complicated, and frequent patterns might be sampled based on any arbitrary target distribution. Then, we propose sampling techniques that are based on more interesting/informative measures or those that are specific to large single networks, such as product of the pattern size with its support, network compressibility, and pattern density. Finally, we study the empirical behavior of our algorithm in a real-world case study.

Aiming at the problems existing in the Chinese-Khmer parallel corpus, such as single field, small scale, and poor timeliness, a method of Chinese-Khmer parallel fragment extraction from comparable corpus based on Dirichlet process is proposed. The method firstly obtains the topic distribution of bilingual comparable corpus through bilingual topic model, then uses Poisson distribution to randomly divide the bilingual texts, sets up a threshold to initially filter parallel fragment of comparable corpus, and then obtains the matching probability between parallel fragments by Dirichlet process. Obtaining the final parallel fragments by Gibbs sampling. From the comparison experiments, the method of parallel fragment extraction from bilingual comparable corpus based on Dirichlet process can obtain higher quality parallel fragments without providing any parallel data, which is more suitable for languages where bilingual resources are low.

Point-of-Interest (POI) recommendation is one of the most essential tasks in LBSNs to help users discover new interesting locations, especially when users travel out of town or to unfamiliar areas. Current studies on POI recommendation in LBSNs mainly focus on modeling multiple factors extracted from users’ profiles and checking-in records. Data sparsity and incompleteness of user-POI interaction matrix are very common problems in POI recommendation, especially for the out-of-town scenario. Another challenge is that most information in the LBSNs is unreliable due to users’ different backgrounds or preferences. Because of the close relationship between users, information from trustable friends on Communication-Based Social Networks (CBSNs) is more valuable than that in LBSNs, which can give a preferable suggestion instead of trustless reviews in LBSNs. In this study, we propose a latent probabilistic generative model called HI-LDA (Heterogeneous Information based LDA), which can accurately capture users’ words on CBSNs by taking into full consideration the information on LBSNs including geographical effect as well as the abundant information including social relationship, users’ interactive behaviors and comment content. In particular, the parameters of the HI-LDA model can be inferred by the Gibbs sampling method in an effective fashion. Beyond these proposed techniques, we introduce an POI recommendation framework integrating geographical clustering approach considering the locations and popularity of POIs simultaneously. Extensive experiments were conducted to evaluate the performance of the proposed framework on two real heterogeneous LBSN-CBSN networks. The experimental results demonstrate the superiority of HI-LDA on effective and efficient POI recommendation in both home-town and out-of-town scenarios, when compared with the state-of-the-art baseline approaches.

Generalized Rayleigh distribution is an important continuous life distribution, which plays a key role in reliability engineering research. In this paper, the statistical inference of reliability of Generalized Rayleigh distribution is studied based on the progressively type-II censored data. First we obtain the estimates through maximum likelihood estimation. And by using Bootstrap methods (Boot-p and Boot-t), we estimate the confidence intervals of parameters and reliability. In addition, based on the square error loss function, the Bayes estimation of the reliability function is carried out by adopting the Gibbs sampling method, and the point estimation, the highest posterior density ( H P D ) confidence interval estimation are carried out respectively. A numerical simulation and an authentic example are used to illustrate the proposed inference methods. Finally, the approximate estimation of stress–strength reliability is further studied.

Incomplete categorical data often occur in the fields such as biomedicine, epidemiology, psychology, sports and so on. In this paper, we first introduce a novel minorization–maximization (MM) algorithm to calculate the maximum likelihood estimates (MLEs) of parameters and the posterior modes for the analysis of general incomplete categorical data. Although the data augmentation (DA) algorithm and Gibbs sampling as the corresponding stochastic counterparts of the expectation–maximization (EM) and ECM algorithms are developed very well, up to now, little work has been done on creating stochastic versions to the existing MM algorithms. This is the first paper to propose a mode-sharing method in Bayesian computation for general incomplete categorical data by developing a new acceptance–rejection (AR) algorithm aided with the proposed MM algorithm. The key idea is to construct a class of envelope densities indexed by a working parameter and to identify a specific envelope density which can overcome the four drawbacks associated with the traditional AR algorithm. The proposed mode-sharing based AR algorithm has three significant characteristics: (I) it can automatically establish a family of envelope densities { g λ ( ⋅ ) : λ ∈ S λ } indexed by a working parameter λ , where each member in the family shares mode with the posterior density; (II) with the one-dimensional grid method searching over the finite interval S λ , it can identify an optimal working parameter λ opt by maximizing the theoretical acceptance probability, yielding a best easy-sampling envelope density g λ opt ( ⋅ ) , which is more dispersive than the posterior density; (III) it can obtain the optimal envelope constant c opt by using the mode-sharing theorem (indicating that the high-dimensional optimization can be completely avoided) or by using the proposed MM algorithm again. Finally, a toy model and three real data sets are used to illustrate the proposed methodologies.

Rapid advancement in machine learning has expedited computer vision-based research applicable to traffic analysis. A 2-stage inference process has been proposed in this paper to learn data distributions applicable to object motion modeling and path learning. In the first stage, a posterior probability learning has been used to get the initial clusters. In the subsequent stage, we use an inference method for likelihood learning by introducing a velocity parameter. It decides the speed at which the model converges to obtain the final clusters. A new sampling method has been proposed that performs better as compared to the Gibbs sampling in terms of computation time. The results demonstrate that the technique has relevance in computer vision applications. The proposed method performs better than the state-of-the-art unsupervised learning methods.

Traffic collision data are always collected in irregularly spaced and mixed frequency. Conventional treatment on these kinds of data, for instance, aggregating the high-frequency data into the lower frequency, can lead to the loss of relevant information of high-frequency data, and introduce potential temporal instabilities. A novel Bayesian vector autoregression approach is proposed to address this problem. An unevenly-spaced traffic collision data with missing values, containing all collisions in different severities that occurred on the state highways in Washington State from January 2006 to December 2016, is selected in this study the impacts of transportation-, weather- and socioeconomic-related characteristics on traffic collisions. A Gibbs sampler is used to conduct Bayesian inference for model parameters and unobserved high-frequency variables. Results show that the model has a fairly superior fit accuracy, and is able to capture the unobserved heterogeneity in the dataset. The proposed VAR also demonstrates better performance than other missing value imputation techniques, including linear regression, predictive mean matching, k-nearest neighbors, and random forests. This study provides potential in the guidance of model construction that considers the mixed-time-series nature of data.

In this paper, objective Bayesian method is applied to analyze degradation model based on the inverse Gaussian process. Noninformative priors (Jefferys prior and two reference priors) for model parameters are obtained and their properties are discussed. Moreover, we propose a class of modified reference priors to remedy weaknesses of the usual reference priors and show that the modified reference priors not only have proper posterior distributions but also have probability matching properties for model parameters. Gibbs sampling algorithms for Bayesian inference based on the Jefferys prior and the modified reference priors are studied. Simulations are conducted to compare the objective Bayesian estimates with the maximum likelihood estimates and subjective Bayesian estimates and shows better performance of the objective method than the other two estimates especially for the case of small sample size. Finally, two real data examples are analyzed for illustration.

In this paper, the objective Bayesian method is applied to investigate the competing risks model involving both catastrophic and degradation failures. By modeling soft failure as the Wiener degradation process, and hard failures as a Weibull distribution, we obtain the noninformative priors (Jefferys prior and two reference priors) for the parameters. Moreover, we show that their posterior distributions have good properties and we propose Gibbs sampling algorithms for the Bayesian inference based on the Jefferys prior and two reference priors. Some simulation studies are conducted to illustrate the superiority of objective Bayesian method. Finally, we apply our methods to two real data examples and compare the objective Bayesian estimates with the other estimates.

A latent Dirichlet allocation (LDA) model is a machine learning technique to identify latent topics from text corpora within a Bayesian hierarchical framework. Current popular inferential methods to fit the LDA model are based on variational Bayesian inference, collapsed Gibbs sampling, or a combination of these. Because these methods assume a unimodal distribution over topics, however, they can suffer from large bias when text corpora consist of various clusters with different topic distributions. This paper proposes an inferential LDA method to efficiently obtain unbiased estimates under flexible modeling for heterogeneous text corpora with the method of partial collapse and the Dirichlet process mixtures. The method is illustrated using a simulation study and an application to a corpus of 1300 documents from neural information processing systems (NIPS) conference articles during the period of 2000–2002 and British Broadcasting Corporation (BBC) news articles during the period of 2004–2005.

Modeling complex relationships and interactions between variables is an ongoing statistical challenge. In particular, the joint modeling of multiple response variables is of great interest in methodological and applied research. Within this context the incorporation of semiparametric predictors into Bayesian recursive simultaneous equation models is considered. Extending the existing framework by imposing effect priors that account for potential correlation of the effects across equations allows for borrowing strength across equations as with multivariate conditional autoregressive priors used for the analysis of multivariate spatial data. A Gibbs sampler is implemented for the estimation and evaluated in an elaborate simulation study where the intra-equation correlation allows for more efficient posterior estimation. The applicability of the novel modeling approach is illustrated with real data examples on malnutrition in Asia and Africa as well as the analysis of plant and species richness with respect to environmental diversity.

This stand-alone tutorial gives an introduction to the QTest 2.1 public domain software package for the specification and statistical analysis of certain order-constrained probabilistic choice models. Like its predecessors, QTest 2.1 allows a user to specify a variety of probabilistic models of binary responses and to carry out state-of-the-art frequentist order-constrained hypothesis tests within a Graphical User Interface (GUI). QTest 2.1 automatizes the mathematical characterization of so-called “random preference models”, adds some parallel computing capabilities, and, most importantly, adds tools for Bayesian inference and model selection. In this tutorial, we provide an in-depth introduction to the Bayesian features: We review order-constrained Bayesian p -values, DIC and Bayes factors, building on the data, models, and prior QTest based frequentist data analyses of an earlier (frequentist) tutorial by Regenwetter et al. (2014).

Bayesian nonparametric (BNP) models are becoming increasingly important in psychology, both as theoretical models of cognition and as analytic tools. However, existing tutorials tend to be at a level of abstraction largely impenetrable by non-technicians. This tutorial aims to help beginners understand key concepts by working through important but often omitted derivations carefully and explicitly, with a focus on linking the mathematics with a practical computation solution for a Dirichlet Process Mixture Model (DPMM)—one of the most widely used BNP methods. Abstract concepts are made explicit and concrete to non-technical readers by working through the theory that gives rise to them. A publicly accessible computer program written in the statistical language R is explained line-by-line to help readers understand the computation algorithm. The algorithm is also linked to a construction method called the Chinese Restaurant Process in an accessible tutorial in this journal (Gershman & Blei, 2012). The overall goals are to help readers understand more fully the theory and application so that they may apply BNP methods in their own work and leverage the technical details in this tutorial to develop novel methods.

Uncovering hidden change-points in an observed signal sequence is challenging both mathematically and computationally. We tackle this by developing an innovative methodology based on Markov chain Monte Carlo and statistical information theory. It consists of an empirical Bayesian information criterion (emBIC) to assess the fitness and virtue of candidate configurations of change-points, and a stochastic search algorithm induced from Gibbs sampling to find the optimal change-points configuration. Our emBIC is derived by treating the unknown change-point locations as latent data rather than parameters as is in traditional BIC, resulting in significant improvement over the latter which is known to mostly over-detect change-points. The use of the Gibbs sampler induced search enables one to quickly find the optimal change-points configuration with high probability and without going through computationally infeasible enumeration. We also integrate the Gibbs sampler induced search with a current BIC-based change-points sequential testing method, significantly improving the method’s performance and computing feasibility. We further develop two comprehensive 3-step computing procedures to implement the proposed methodology for practical use. Finally, simulation studies and real examples analyzing business and genetic data are presented to illustrate and assess the procedures.

The aim of this study was to investigate the genetic and non-genetic effects on survival traits of Sangsari sheep from birth to the last recording date. Data base with 35,470 sheep survival records was collected during 1988 to 2010 in 11 flocks by the Animal Breeding Center of Iran. The non-genetic effects were evaluated on hazard rate of sheep in different times with survival and cmprsk packages. Exponential distribution based on censored data was used to estimate the genetic parameters of the survival from birth to the last recording date. The linear animal models were fitted using Bayesian method via Gibbs sampling of the MCMCglmm package. The overall mortality rate for Sangsari lambs was 2.00 and 25.72% for birth to 100 and 500 day of age, respectively, and its value increased upto 90.54% till 3400 day of age. The results indicated that the year and month of birth, sex of lamb, birth weight, age of ewe, flock (P

In the present study, 43,441 body weight-age records (21,361 records on male lambs and 22,060 records on female lambs) on 6899 lambs descended from 235 sires and 2842 dams that collected from 1991 to 2015 in Kordi sheep breeding station, located in Shirvan city, North Khorasan province, North-eastern area of Iran, were used. Five non-linear models including Brody, Negative exponential, von Bertalanffy, Logistic and Gompertz were fitted on body weight-age data and compared applying Akaike's information criterion (AIC), mean square error (MSE) and adjusted coefficient of determination (R2adg) for determining the most appropriate model describing the growth curve for all, male and female Kordi lambs. The Brody model showed the lowest AIC and MSE and the highest R2adg among the tested models for all, male and female Kordi lambs. Therefore, Brody model was selected as the best one for describing growth curve in Kordi sheep. The estimated values for parameters of growth curve under Brody model including parameters A (estimated mature weight), B (an integration constant related to initial animal weight) and K (maturation rate) were 43.372, 0.905 and 0.007, respectively. Parameters of A, B and K were estimated as 48.630, 0.911 and 0.006, respectively for male lambs and 38.444, 0.898 and 0.008 for female lambs, respectively. A nultivariate animal model was considered for genetic analysis of the growth curve parameters via Gibbs sampling under a Bayesian approach with 100,000 iterations, considering the first 20,000 samples as the burn-in period and a thinning interval of 10 samples. Posterior means for heritability estimates of the studied growth curve parameters of Kordi sheep were 0.08, 0.14 and 0.04 for parameters A, B and K, respectively; showed low level of genetic variation for the parameters of growth curve in Kordi sheep. Posterior means for the estimates of genetic correlations among the studied growth curve traits were 0.48 (A-B), -0.22 (A-K) and -0.28 (B-K). The Brody model can be considered for predicting live weight of Kordi lambs at later ages from early body weight data and can help planning farm management practices such as feeding and drug administration (since these practices are commonly focused on live body weight) and also decision making regarding the selecting of highly productive animals by monitoring their growth curve.

In this paper, we propose new Bayesian hierarchical representations of lasso, adaptive lasso and elastic net quantile regression models. We explore these representations by observing that the lasso penalty function corresponds to a scale mixture of truncated normal distribution (with exponential mixing densities). We consider fully Bayesian treatments that lead to new Gibbs sampler methods with tractable full conditional posteriors. The new methods are then illustrated with both simulated and real data. Results show that the new methods perform very well under a variety of simulations, such as the presence of a moderately large number of predictors, collinearity and heterogeneity.

This paper proposes a nonparametric Bayesian approach using a Dirichlet process mixture model with Pareto kernels to estimate the density of observed upper record values and predict future upper record values. A reference distribution, G0, in the nonparametric Bayesian model is provided on the basis of an objective prior for unknown parameters of the Pareto distribution to avoid difficulties caused by finding values of hyperparameters in G0. For the posterior computation, two sampling algorithms are employed and compared. Finally, the approach provides the estimated distribution of observed upper record values and prediction of future upper record values in two real problems, average annual temperatures and carbon dioxide emissions.

The traditional collaborative filtering (CF) suffers from two key challenges, namely, the normal assumption that it is not robust, and it is difficult to set in advance the penalty terms of the latent features. We therefore propose a hierarchical Bayesian model-based CF and the related inference algorithm. Specifically, we impose a Gaussian-Gamma prior on the ratings, and the latent features. We show the model is more robust, and the penalty terms can be adapted automatically in the inference. We use Gibbs sampler for the inference and provide a statistical explanation. We verify the performance using both synthetic and real datasets

Bayesian variable selection is an active research area. The paper proposes a variable selection technique by taking advantages of two distinctive approaches of Bayesian variable selection, namely spike and slab prior and nonlocal prior. Contrary to the local priors, nonlocal priors put zero mass at null values of the parameters. The proposed method uses posterior median as the parameter estimates and asymptotic consistency is established. The Bayesian implementation is proposed via Gibbs sampling. The full conditional distributions are provided. Extensive numerical study indicates superiority of the proposed procedure compared to its competitors. Most importantly, the procedure works well in low signal situation which is not the case in general.

Latent variable hidden Markov models (LVHMMs) are important statistical methods in exploring the possible heterogeneity of data and explaining the pattern of subjects moving from one group to another over time. Classic subject- and/or time-homogeneous assumptions on transition matrices in transition model as well as the emission distribution in the observed process may be inappropriate to interpret heterogeneity at the subject level. For this end, a general extension of LVHMM is proposed to address the heterogeneity of multivariate longitudinal data both at the subject level and the occasion level. The main modeling strategy is that the observed time sequences are first grouped into different clusters, and then within each cluster the observed sequences are formulated via latent variable hidden Markov model. The local heterogeneity at the occasion level is characterized by the distribution related to the latent states, while the global heterogeneity at the subject level is identified with the finite mixture model. Compared to the existing methods, an appeal underlying the proposal is its capacity of accommodating non-homogeneous patterns of state sequences and emission distributions across the subjects simultaneously. As a result, the proposal provides a comprehensive framework for exploring various kinds of relevance among the multivariate longitudinal data. Within the Bayesian paradigm, Markov Chains Monte Carlo (MCMC) method is used to implement posterior analysis. Gibbs sampler is used to draw observations from the related full conditionals and posterior inferences are carried out based on these simulated observations. Empirical results including simulation studies and a real example are used to illustrate the proposed methodology.

In topic modeling framework, many Dirichlet-based models performances have been hindered by the limitations of the conjugate prior. It led to models with more flexible priors, such as the generalized Dirichlet distribution, that tend to capture semantic relationships between topics (topic correlation). Now these extensions also suffer from incomplete generative processes that complicate performances in traditional inferences such as VB (Variational Bayes) and CGS (Collaspsed Gibbs Sampling). As a result, the new approach, the CVB-LGDA (Collapsed Variational Bayesian inference for the Latent Generalized Dirichlet Allocation) presents a scheme that integrates a complete generative process to a robust inference technique for topic correlation and codebook analysis. Its performance in image classification, facial expression recognition, 3D objects categorization, and action recognition in videos shows its merits.

We investigate how ideas from covariance localization in numerical weather prediction can be used in Markov chain Monte Carlo (MCMC) sampling of high-dimensional posterior distributions arising in Bayesian inverse problems. To localize an inverse problem is to enforce an anticipated “local” structure by (i) neglecting small off-diagonal elements of the prior precision and covariance matrices; and (ii) restricting the influence of observations to their neighborhood. For linear problems we can specify the conditions under which posterior moments of the localized problem are close to those of the original problem. We explain physical interpretations of our assumptions about local structure and discuss the notion of high dimensionality in local problems, which is different from the usual notion of high dimensionality in function space MCMC. The Gibbs sampler is a natural choice of MCMC algorithm for localized inverse problems and we demonstrate that its convergence rate is independent of dimension for localized linear problems. Nonlinear problems can also be tackled efficiently by localization and, as a simple illustration of these ideas, we present a localized Metropolis-within-Gibbs sampler. Several linear and nonlinear numerical examples illustrate localization in the context of MCMC samplers for inverse problems. Keyword: Markov chain Monte Carlo; Bayesian inverse problems; high dimensions; localization; dimension-independent convergence, United States. Department of Energy. Office of Advanced Scientific Computing Research (Grant DE-SC0009297)

Recently, automatic feature extraction and selection from unlabeled images that contain irrelevant patterns have been a proceeding interest. In this paper, an enhanced high-order Boltzmann machine is designed to promote the capacity of feature extraction and selection in a unified context. First, gating mechanism is employed for feature selection in comparison with conventional approaches. Then, two sets of hidden variables that the one set is real-valued latent variables and the other is spike latent variables are introduced to model the covariance structure of local patches, which can boost the abilities of feature learning and feature selection in turn. Simultaneously, the proposed model can infer in parallel via easy block Gibbs sampling without much training difficulty. Last, several extensions of the proposed model are developed to cope with different scenes. The massive performances obtained from various visual tasks have demonstrated that the proposed model can reach the highly improved performances over several currently excellent methods.

Many countries around the world use a centralized admission system for assigning thousands of students to academic programs in universities every year. That said, little is known about how changes in admission policies impact students’ welfare and allocations. To do so, policy makers would need to infer students’ true preferences from their potentially manipulated reports. We propose a novel methodology to recover students’ true preferences in a mechanism where students have private information and are constrained in the number of options they can submit as preferences, and, thus, when they have an incentive to report strategically. The proposed methodology consists of two steps. In the first step, we are able to non-parametrically recover the distribution of ordinal preferences by using a revealed preference approach that involves the implementation of a concatenation algorithm. In the second step, we parametrically recover the distribution of cardinal preferences by using a Gibbs Sampler estimator. We use a rich administrative dataset from the University of Costa Rica to implement our methodology and evaluate several counterfactual policies. Finally, we propose two alternate mechanisms based on an ascending auction and on prices of competitive equilibrium.

This tutorial provides an introduction to Bayesian modeling and Markov Chain Monte-Carlo (MCMC) algorithms including the Metropolis-Hastings and Gibbs Sampling algorithms. We discuss some of the challenges associated with running MCMC including the important question of determining when convergence to stationarity has been achieved. Several applications of Bayesian modeling are also provided including the MRP approach to modeling election outcomes, topic modeling in machine learning, and large-scale optimization and code breaking. We also discuss the important problems of Bayesian model checking and selection and provide an introduction to empirical Bayesian models. In the appendices we briefly discuss several topics including: (i) Hamiltonian Monte-Carlo (an MCMC algorithm that forms the basis for the statistical modeling language STAN) (ii) the EM algorithm and (iii) other problems in Bayesian statistics including Bayesian approaches to constructing confidence intervals, hypothesis testing and decision theory.

Flotation tests at laboratory scale describe the metallurgical behavior of the minerals that will be processed in the operational plant. This material is generally composed of ore and gangue minerals. These tests are usually scarce, expensive and sampled in large supports. This research proposes a methodology for the geostatistical modelling of metallurgical recovery, covering the change of support problems through additive auxiliary variables. The methodology consists of simulating these auxiliary variables using a Gibbs Sampler in order to infer the behavior of samples with smaller supports. This allows downscaling a large sample measurement into smaller ones, reproducing the variability at different scales considering the physical restrictions of additivity balance of the metallurgical recovery process. As a consequence, it is possible to apply conventional multivariate geostatistical tools to data at different supports, such as multivariable exploratory analysis, calculation of cross-variograms, multivariate estimations, among others. The methodology was tested using a drillhole database from an ore deposit, modelling recovery at a smaller support than that of the metallurgical tests. The support allowed for the use of the geochemical database, to consistently model the metal content in the feed and in the concentrate, in order to obtain a valid recovery model. Results show that downscaling the composite size reduces smoothing in the final model.

A major part of the growth performance in birds is the result of the combined effects of genes, including the general and specific combining ability that requires the design of an optimal mating system. The aim of this study was to fit the best model for each body weight trait from hatch to 45 days and the estimation of the variance components and the genetic parameters for the body weight traits of a crossbred population. This crossbred population was created by 4 strains of Japanese quail, including the Italian Speckled (A), Tuxedo (B), Pharaoh (C), Texas A&M (D), and the body weights of the different combinations were analyzed by 24 models including the direct genetic effect, the non-additive genetic effects including dominance and epistatic effects, the maternal permanent environmental and maternal genetic effects. The selection of the best fit model of each trait was performed based on the deviance information criteria. The variance components were estimated using a single-trait animal model analyzed with Gibbs sampling. At the early stage of bird growth, maternal genetic and maternal permanent environmental effects had a considerable contribution to the best model, but the contribution of these effects reduced with an increase in the bird's age and the additive variance contribution increased. Adding non-additive genetic effects (dominance and epistasis) to the models significantly reduced the variance of the error and the additive genetic variance estimated with high accuracy. The estimated heritability of body weight traits for the body weights of hatch, 5, 10, 15, 20, 25, 30, 35, 40, and 45 d were 0.316, 0.170, 0.251, 0.153, 0.132, 0.164, 0.290, 0.425, 0.476, and 0.362, respectively. The ratio of maternal genetic and maternal permanent environmental was considerable especially on early age body weight but the ratio of dominance and epistatic variances on each of the body weight traits was less than 4.5% of the total variance, but led to a more accurate estimates of the direct additive genetic.

International audience; A potentially sensitive way to detect disease outbreaks is syndromic surveillance, i.e. monitoring the number of syndromes reported in the population of interest, comparing it to the baseline rate, and drawing conclusions about outbreaks using statistical methods. A decision maker may use the results to take disease control actions or to initiate enhanced epidemiological investigations. In addition to the total count of syndromes there are often additional pieces of information to consider when assessing the probability of an outbreak. This includes clus-tering of syndromes in space and time as well as historical data on the occurrence of syndromes, seasonality of the disease, etc. In this paper, we show how Bayesian theory for syndromic surveillance applies to the occurrence of neurological syndromes in horses in France. Neurological syndromes in horses may be connected e.g. to West Nile Virus (WNV), a zoonotic disease of growing concern for public health in Europe. A Bayesian method for spatio-temporal cluster detection of syndromes and for determining the probability of an outbreak is presented. It is shown how surveillance can be performed simultaneously for a specific class of diseases (WNV or diseases similar to WNV in terms of the information available to the system) and a non-specific class of diseases (not similar to WNV in terms of the information available to the system). We also discuss some new extensions to the spatio-temporal models and the computational algorithms involved. It is shown step-by-step how data from historical WNV outbreaks and surveillance data for neurological syndromes can be used for model construction. The model is implemented using a Gibbs sampling procedure, and its sensitivity and specificity is evaluated. Finally, it is illustrated how predictive modelling of syndromes can be useful for decision making in animal health surveillance.

Genetic parameters for reproductive traits in Baluchi sheep were estimated on 3844 dams born during 1991 to 2012. The studied traits were litter size at birth (LSB), litter size at weaning (LSW), litter mean weight per lamb born (LMWLB), litter mean weight per lamb weaned (LMWLW), total litter weight at birth (TLWB) and total litter weight at weaning (TLWW). Significant effects were determined using Logistic and GLM procedure of SAS software for threshold and continuous traits, respectively. Variance components needed for breeding were estimated using Gibbs sampling methodology of Bayesian inference. The genetic parameters for LMWLB, LMWLW, TLWB TLWW were estimated by fitting a linear model. To examine the appropriateness of fitting linear or threshold model to litter size at birth and weaning, both models were used to estimate genetic parameters on the observed data. Posterior means of the heritabilities of litter size at birth and weaning by using a threshold model were estimated to be two times higher than the heritabilities based on the linear model (0.18 versus 0.09 and 0.11 versus 0.05 for LSB and LSW, respectively). Bayesian information criterion (BIC) selected the threshold model as most appropriate model for LSB and LSW given the data. The heritability estimates for LMWLB, LMWLW, TLWB TLWW resulting from linear analysis were 0.12, 0.05, 0.06 and 0.08, respectively. Genetic correlation estimates between the traits ranged from −0.67 for LMWLB–TLWW to 0.85 for LSB–TLWB. Phenotypic correlations ranged from −0.06 for LSB–LMWLW to 0.88 for LSB–TLWB and permanent environmental correlations ranged from −0.74 for LMWLB - TLWW to 0.88 for LSB–TLWB. Results of this study showed that the threshold model seems to be reliable for genetic evaluation of litter size in Baluchi sheep.

We use data from the Swiss national household travel survey to 1. analyze the socioeconomic determinants for intermodal travel in Switzerland and 2. estimate a first large-scale multimodal recursive logit route choice model for urban trip making. We show that intermodal travel is mostly associated with ownership of transit subscriptions, which allow free at the point-of-use public transportation. We also present a framework using open-source data to generate the multimodal network for the recursive logit model estimation. The fact that the model only needs a multimodal network to model the transport supply makes it independent of path sampling algorithms for the choice-set construction and it thus provides an alternative to classic mode and route choice models, since it can estimate mode and route choice parameters with directly observed routes, removing the sampling bias. By eliminating the need to sample alternative paths for estimation, it also simplifies the estimation process, making it a viable choice as an integral solution for joint route and mode choice modelling.

Runoff is an important basis for the operation of hydropower. However, due to inflow uncertainty, the ideal practical operation process usually doesn't match with the plan, which exacerbates the power generation and ecological risks. Therefore, the purpose of this paper is to investigate the relationship between hydropower generation, ecology and their risks under the uncertainty of inflow. Gibbs sampling based on the copula function is presented to simulate the runoff. The scenario tree method is employed to describe the inflow uncertainty. The power generation risk operation model is developed based on the Mean-variance method. In addition, the expected minimum ecological risk model and its comparison model are proposed to analyze the relationship between power generation and ecological risk. The proposed methods are applied to a case study of the Lancang River cascade hydropower station. The results show that (1) The power generation risk of risk operation model decreased by 86.41% at the 90% scenario reduction level, compared with deterministic model. (2) There is an obvious competitive relationship between ecology and power generation, in the case of a 1% loss of expected power generation, the expected ecological risk can be reduced by 6.14%.

To model spatial point patterns with discrete time stamps a flexible spatio-temporal area-interaction point process is proposed. In particular, this model is suitable for describing the dependency between point patterns over time, when the new point pattern arises from the previous point pattern. A hierarchical model is also implemented in order to incorporate the underlying evolution process of the model parameters. For parameter estimation, a double Metropolis-Hastings within Gibbs sampler is used. The performance of the estimation algorithm is evaluated through a simulation study. Finally, the point pattern forecasting procedure is demonstrated through a simulation study and an application to United States natural caused wildfire data from 2002 to 2019.

Ensembles of geological realizations (GR) are normally processed by numerical simulators to evaluate geological uncertainty during the decision-making process. Although different stochastic spatial algorithms can quickly generate hundreds to thousands of GR to capture the full uncertainty range, the simulation process applied to this number of realizations is computationally expensive. Hence, a small subset of representative geological realizations (RGR) that statistically represent the features of the full ensemble can be used for uncertainty quantification. In this study, unsupervised machine learning (UML) is applied by considering different (1) adjacency matrix construction, (2) dimensionality reduction, and (3) clustering and sampling algorithms to generate several RGR sets. Then, the mismatches between the distribution of different field and well indicators obtained from the RGR sets and the whole ensemble are measured using the Kolmogorov-Smirnov (KS) test to compare the uncertainty space of the subsets and the full set. Furthermore, to measure the pairwise adjacency between the realizations, we use a static reservoir feature called reservoir quality index (RQI). We performed extensive computational analyses to appraise the performance of the UML in two benchmark cases. Each case contains 500 GR. This study can provide a comprehensive assessment of the UML for the RGR selection due to the application of different algorithms. The results showed that the RGR set can be successfully selected without previous flow simulation runs, if an appropriate UML method is employed. This leads to a reduction in the computational cost during uncertainty quantification and risk analysis. Furthermore, we observed that the optimal number of RGR should be chosen due to the geological complexity of each case study. We also found that the type of recovery mechanism has no impact on the optimal number of RGR and on UML methods. The appropriate RGR set can be used for production forecasts and development planning support.

Bayesian inference methods typically require a considerable amount of computation time in the calculation of forward models. This limitation restricts the application of Bayesian inference methods for the parameter identification of complex engineering problems. We propose a novel likelihood-free Bayesian inference method for structural parameter identification. An adaptive Gaussian surrogate model (GSM) was integrated with the transitional Markov chain Monte Carlo (MCMC) method for Bayesian inference. The log-likelihood function was approximated with GSM and the transitional MCMC method was used to generate the posterior distribution samples. A response reconstruction technique was combined with the likelihood-free Bayesian inference method for the parameter identification. Both numerical studies and experimental studies were conducted to verify the accuracy and efficiency of the proposed method. The results showed that the proposed method could be used to estimate the posterior probabilities of unknown structural parameters. Additionally, the proposed method was more efficient than the delayed rejection adaptive Metropolis and Gibbs sampling methods.

In reliability or warranty analysis, engineers must often deal with lifetimes data that are non-homogeneous. Most of the time, this variability is unobserved but has to be taken into account for reliability or warranty cost analysis. A further problem is that in reliability analysis, data are heavily censored which makes estimations more difficult. The two-component Weibull mixture is then a highly relevant model to capture heterogeneity for a large majority of operating lifetimes. Unfortunately, the performance of classical estimation methods (maximum of likelihood via EM, Bayes approach via MCMC) is jeopardized due to the high number of parameters and the heavy censoring. In order to overcome the problem of heavy censoring for Weibull mixture parameters estimation, this research proposes a Bayesian bootstrap method, called Bayesian Restoration Maximization. The key is to provide a sampling from the posterior distribution. Thanks to an importance sampling technique, this sample focuses on the posterior mean. Prior distributions elicitation and sensibility analysis are discussed. Simulations results showed that, for heavily censored data, the BRM method outperforms the EM and S-EM algorithms in terms of estimates accuracy. On the other hand, it is a non-iterative method which therefore provides very short computation times. In addition, the BRM method does not suffer from the problem of label switching from Bayesian sampling algorithms such as MCMC methods. Finally, two real data sets are analyzed to illustrate the application of the method.

We present a joint Bayesian inverse method for analyzing conservative and reactive tracer data, which estimates non-parametric transfer functions and effective first-order reaction rate coefficients simultaneously. A stochastic sampling approach based on Markov Chain Monte Carlo (MCMC) methods was used to simulate samples from the posteriors, with a Metropolis within Gibbs sampler used for transfer functions and reaction rate coefficients. Non-negativity constraints were realized by choosing a reflected Gaussian prior for the transfer function and a lognormal prior for the reaction rate coefficient. The approach was tested by synthetic sparse datasets with introduced normal random errors or periodic errors, and then it was applied to the tracer test data collected at Oak Ridge National Laboratory. The results indicate that the developed approach was well capable of characterizing anomalous transfer functions, such as multi-modal functions, and quantifying the uncertainty of both transfer functions and reaction rate coefficients. In particular, compared with the approach of individual inversion of sparse data, the joint inversion approach taking advantage of the sharing of information regarding transport processes across the conservative tracer and reactive tracer data leads to the improved estimation of transfer functions and reaction rate coefficients for sparse datasets.

Background and ObjectiveBayesian State Space models are recent advancement in stochastic modeling which capture the randomness of a hidden background process by scrutinizing the prior knowledge and likelihood of observed data. This article elucidate the scope of Bayesian state space modeling on predicting the future expression values of a longitudinal micro array data. MethodsThe study conveniently makes use of longitudinally collected clinical trial data (GSE30531) from NCBI Gene Expression Omnibus (GEO) data repository. Multiple testing methodology using t-test is used for selecting differentially expressed genes between groups for fitting the model. The parameter values of the predictive model and future expression levels are estimated by drawing samples from the posterior joint distribution using a stochastic Markov Chain Monte Carlo (MCMC) algorithm which relies on Gibbs Sampling. The study also made an attempt to get estimates and its 95% Credible Interval through assumptions of different covariance structures like Variance Components, First order Auto Regressive and Unstructured variance–covariance structure to showcase the flexibility of the algorithm. Results72 Distinct genes with significantly different expression levels where selected for model fitting. Parameter estimates showed almost similar trends under different covariance structure assumption. Cross tabulation of gene frequencies having minimum credible interval under each covariance structure and study group showed a significant P value of 0.02. ConclusionsPresent study reveals that Bayesian state space models can be effectively used to explain and predict a complex data like gene expression data.

The phenotypic plasticity of cancer cells has received special attention in recent years. Even though related models have been widely studied in terms of mathematical properties, a thorough statistical analysis on parameter estimation and model selection is still very lacking. In this study, we present a Bayesian approach which is devised to deal with the data sets containing both mean and variance information of relative frequencies of cancer stem cells (CSCs). Both Gibbs sampling and Metropolis-Hastings (MH) algorithm are used to perform point and interval estimations of cell-state transition rates between CSCs and non-CSCs. Extensive simulations demonstrate the validity of our model and algorithm. By applying this method to a published data on SW620 colon cancer cell line, the model selection favors the phenotypic plasticity model, relative to conventional hierarchical model of cancer cells. Further quantitative analysis shows that, in the presence of phenotypic equilibrium, the variance data greatly influences the time-variant pattern of the parameters. Moreover, it is found that the occurrence of self-renewal of CSCs shows a strong negative correlation with de-differentiation rate from non-CSCs to CSCs, suggesting a balancing mechanism in the heterogenous population of cancer cells.

Expert systems routinely use conditional reasoning. Conditionally specified statistical models offer several advantages over joint models; one is that Gibbs sampling can be used to generate realizations of the model. As a result, full conditional specification for multiple imputation is gaining popularity because it is flexible and computationally straightforward. However, it would be restrictive to require that every regression/classification must involve all of the variables. Feature selection often removes some variables from the set of predictors, thus making the regression local. A mixture of full and local conditionals is referred to as a partially collapsed Gibbs sampler, which often achieves faster convergence due to reduced conditioning. However, its implementation requires choosing a correct scan order. Using an invalid scan order will bring about an incorrect transition kernel, which leads to the wrong stationary distribution. We prove a necessary and sufficient condition for Gibbs sampling to correctly sample the joint distribution. We propose an algorithm that identifies all of the valid scan orders for a given conditional model. A forward search algorithm is discussed. Checking compatibility among conditionals of different localities is also discussed.

Active classification systems developed by machine learning researchers have helped to recognize and retrieve object categories more rapidly. However, a large number of studies have focused on designing feedback procedures or sampling algorithms that maximize the information contained in individual examples for training the classifier, but it results in increasing time expense with the size of database. In this work, an attribute distance is proposed directly on visual attributes. And then, active classification systems are constructed on the chosen examples in a short time. Our contributions are reported as follows: (1) a small set of visual attributes can be discriminative for current classification tasks, and therefore, it can speed up the computation of example-to-hyperplane distances; (2) discriminative attributes are sequentially observed by using their scores given by the classifier, corresponding to target objects. We apply the attribute distance in several applications and show experimental results of speeding up feedback procedures. Moreover, the attribute distance can bring insight into designing a fast sampling strategy which is able to further improve the efficiency for existing classification systems.

The present work is associated with Bayesian finite element (FE) model updating using modal measurements based on maximizing the posterior probability instead of any sampling based approach. Such Bayesian updating framework usually employs normal distribution in updating of parameters, although normal distribution has usual statistical issues while using non-negative parameters. These issues are proposed to be dealt with incorporating lognormal distribution for non-negative parameters. Detailed formulations are carried out for model updating, uncertainty-estimation and probabilistic detection of changes/damages of structural parameters using combined normal-lognormal probability distribution in this Bayesian framework. Normal and lognormal distributions are considered for eigen-system equation and structural (mass and stiffness) parameters respectively, while these two distributions are jointly considered for likelihood function. Important advantages in FE model updating (e.g. utilization of incomplete measured modal data, non-requirement of mode-matching) are also retained in this combined normal-lognormal distribution based proposed FE model updating approach. For demonstrating the efficiency of this proposed approach, a two dimensional truss structure is considered with multiple damage cases. Satisfactory performances are observed in model updating and subsequent probabilistic estimations, however level of performances are found to be weakened with increasing levels in damage scenario (as usual). Moreover, performances of this proposed FE model updating approach are compared with the typical normal distribution based updating approach for those damage cases demonstrating quite similar level of performances. The proposed approach also demonstrates better computational efficiency (achieving higher accuracy in lesser computation time) in comparison with two prominent Markov Chain Monte Carlo (MCMC) techniques (viz. Metropolis-Hastings algorithm and Gibbs sampling).