Search

We investigate the relationship between count data that inform the relative abundance of features of a composition, and factors that influence the composition. Our work is motivated from microbiome studies aiming to extract microbial signatures that are predictive of host phenotypes based on data collected from a group of individuals harboring radically different microbial communities. We introduce multinomial Factor Augmented Inverse Regression (FAIR) of the count vector onto response factors as a general framework for obtaining low-dimensional summaries of the count vector that preserve information relevant to the response. By augmenting known response factors with random latent factors, FAIR extends multinomial logistic regression to account for overdispersion and general correlations among counts. The projections of the count vector onto the loading vectors represent additional contribution, in addition to the projections that result from response factors. The method of maximum variational likelihood and a fast variational expectation-maximization algorithm are proposed for approximate inference based on variational approximation, and the asymptotic properties of the resulting estimator are derived. Moreover, a hybrid information criterion and a group-lasso penalized criterion are proposed for model selection. The effectiveness of FAIR is illustrated through simulations and application to a microbiome dataset. for this article are available online. [ABSTRACT FROM AUTHOR]

Probabilistic models such as logistic regression, Bayesian classification, neural networks, and models for natural language processing, are increasingly more present in both undergraduate and graduate statistics and data science curricula due to their wide range of applications. In this article, we present a one-week course module for students in advanced undergraduate and applied graduate courses on variational inference, a popular optimization-based approach for approximate inference with probabilistic models. Our proposed module is guided by active learning principles: In addition to lecture materials on variational inference, we provide an accompanying class activity, an R shiny app, and guided labs based on real data applications of logistic regression and clustering documents using Latent Dirichlet Allocation with R code. The main goal of our module is to expose students to a method that facilitates statistical modeling and inference with large datasets. Using our proposed module as a foundation, instructors can adopt and adapt it to introduce more realistic case studies and applications in data science, Bayesian statistics, multivariate analysis, and statistical machine learning courses. [ABSTRACT FROM AUTHOR]

AbstractComputer models play a crucial role in numerous scientific and engineering domains. To ensure the accuracy of simulations, it is essential to properly calibrate the input parameters of these models through statistical inference. While Bayesian inference is the standard approach for this task, employing Markov chain Monte Carlo methods often encounters computational hurdles due to the costly evaluation of likelihood functions and slow mixing rates. Although variational inference (VI) can be a fast alternative to traditional Bayesian approaches, VI has limited applicability due to boundary issues and local optima problems. To address these challenges, we propose flexible VI methods based on deep generative models that do not require parametric assumptions on the variational distribution. We embed a surjective transformation in our framework to avoid posterior truncation at the boundary. Additionally, we provide theoretical conditions that guarantee the success of the algorithm. Furthermore, our temperature annealing scheme and fine-tuning can prevent being trapped in local optima through a series of intermediate posteriors and weight adjustment. We apply our method to infectious disease models and a geophysical model, illustrating that the proposed method can provide fast and accurate inference compared to its competitors. [ABSTRACT FROM AUTHOR]

Networks consist of interconnected units, known as nodes, and allow to formally describe interactions within a system. Specifically, bipartite networks depict relationships between two distinct sets of nodes, designated as sending and receiving nodes. An integral aspect of bipartite network analysis often involves identifying clusters of nodes with similar behaviors. The computational complexity of models for large bipartite networks poses a challenge. To mitigate this challenge, we employ a Mixture of Latent Trait Analyzers (MLTA) for node clustering. Our approach extends the MLTA to include covariates and introduces a double EM algorithm for estimation. Applying our method to COVID-19 data, with sending nodes representing patients and receiving nodes representing preventive measures, enables dimensionality reduction and the identification of meaningful groups. We present simulation results demonstrating the accuracy of the proposed method. [ABSTRACT FROM AUTHOR]

We propose a novel variational Bayes approach to estimate high-dimensional Vector Autoregressive (VAR) models with hierarchical shrinkage priors. Our approach does not rely on a conventional structural representation of the parameter space for posterior inference. Instead, we elicit hierarchical shrinkage priors directly on the matrix of regression coefficients so that (a) the prior structure maps into posterior inference on the reduced-form transition matrix and (b) posterior estimates are more robust to variables permutation. An extensive simulation study provides evidence that our approach compares favorably against existing linear and nonlinear Markov chain Monte Carlo and variational Bayes methods. We investigate the statistical and economic value of the forecasts from our variational inference approach for a mean-variance investor allocating her wealth to different industry portfolios. The results show that more accurate estimates translate into substantial out-of-sample gains across hierarchical shrinkage priors and model dimensions. [ABSTRACT FROM AUTHOR]

AbstractWe propose a new approach to Bayesian prediction that caters for models with a large number of parameters and is robust to model misspecification. Given a class of high-dimensional (but parametric) predictive models, this new approach constructs a posterior predictive using a variational approximation to a generalized posterior that is directly focused on predictive accuracy. The theoretical behavior of the new prediction approach is analyzed and a form of optimality demonstrated. Applications to both simulated and empirical data using high-dimensional Bayesian neural network and autoregressive mixture models demonstrate that the approach provides more accurate results than various alternatives, including misspecified likelihood-based predictions. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

AbstractIn this work, we advocate for the importance of singular learning theory (SLT) as it pertains to the theory and practice of variational inference in Bayesian neural networks (BNNs). To begin, we lay to rest some of the confusion surrounding discrepancies between downstream predictive performance measured via the test log predictive density and the variational objective. Next, we use the SLT-corrected asymptotic form for singular posterior distributions to inform the design of the variational family itself. Specifically, we build upon the idealized variational family introduced in Bhattacharya, Pati, and Plummer which is theoretically appealing but practically intractable. Our proposal takes shape as a normalizing flow where the base distribution is a carefully-initialized generalized gamma. We conduct experiments comparing this to the canonical Gaussian base distribution and show improvements in terms of variational free energy and variational generalization error. Supplemental appendices and code for the article are available online. [ABSTRACT FROM AUTHOR]

Neural demyelination and brain damage accumulated in white matter appear as hyperintense areas on T2-weighted MRI scans in the form of lesions. Modeling binary images at the population level, where each voxel represents the existence of a lesion, plays an important role in understanding aging and inflammatory diseases. We propose a scalable hierarchical Bayesian spatial model, called BLESS, capable of handling binary responses by placing continuous spike-and-slab mixture priors on spatially varying parameters and enforcing spatial dependency on the parameter dictating the amount of sparsity within the probability of inclusion. The use of mean-field variational inference with dynamic posterior exploration, which is an annealing-like strategy that improves optimization, allows our method to scale to large sample sizes. Our method also accounts for underestimation of posterior variance due to variational inference by providing an approximate posterior sampling approach based on Bayesian bootstrap ideas and spike-and-slab priors with random shrinkage targets. Besides accurate uncertainty quantification, this approach is capable of producing novel cluster size based imaging statistics, such as credible intervals of cluster size, and measures of reliability of cluster occurrence. Lastly, we validate our results via simulation studies and an application to the UK Biobank, a large-scale lesion mapping study with a sample size of 40,000 subjects. for this article are available online. [ABSTRACT FROM AUTHOR]

AbstractProfile monitoring is a widely used tool in quality control. The rapid development of sensor technology has created unprecedented opportunities for multi-channel profile data collection, which motivates the modeling and transfer learning for multi-profile data. The Multi-output Gaussian Process (MGP) is often used for multi-profile data, due to its flexible modeling capability and elegant mathematical properties. However, two practical concerns limit the broader application of MGP for transfer learning and monitoring of multi-profile data: high computational complexity and data incompleteness. In this article, we propose a Variational Inference (VI)-based MGP framework to facilitate transfer learning and profile monitoring using incomplete profile data. The proposed framework features a specially designed convolutional structure for constructing an explicit covariance relationship between the inducing variables in VI and the MGP in multi-profile data. This structure inspires a comprehensive solution to both computational complexity and data incompleteness in modeling multi-profile data, which facilities the transfer learning for profile monitoring. Various numerical studies and one real case study are conducted to demonstrate and compare the transfer learning and monitoring performance of the proposed method. The results show the method can achieve superior monitoring performance while maintain a very low level of computational complexity. [ABSTRACT FROM AUTHOR]

Multi-output Gaussian process (MGP) regression plays an important role in the integrative analysis of different but interrelated systems/units. Existing MGP approaches assume that data from all units is collected and stored at a central location. This requires massive computing and storage power at the central location, induces significant communication traffic due to raw data exchange, and comprises privacy of units. However, recent advances in Internet of Things technologies, which have tremendously increased edge computing power, pose a significant opportunity to address such challenges. In this article, we propose FedMGP, a general federated analytics (FA) framework to learn an MGP in a decentralized manner that uses edge computing power to distribute model learning efforts. Specifically, we propose a hierarchical modeling approach where an MGP is built upon shared global latent functions. We then develop a variational inference FA algorithm that overcomes the need to share raw data. Instead, collaborative learning is achieved by only sharing global latent function statistics. Comprehensive simulation studies and a case study on battery degradation data highlight the superior predictive performance and versatility of FedMGP, achieved while distributing computing and storage demands, reducing communication burden, fostering privacy, and personalizing analysis. [ABSTRACT FROM AUTHOR]

AbstractLatent Gaussian models (LGMs) are perhaps the most commonly used class of models in statistical applications. Nevertheless, in areas ranging from longitudinal studies in biostatistics to geostatistics, it is easy to find datasets that contain inherently non-Gaussian features, such as sudden jumps or spikes, that adversely affect the inferences and predictions made using an LGM. These datasets require more general latent non-Gaussian models (LnGMs) that can handle automatically these non-Gaussian features. However, fast implementation and easy-to-use software are lacking, preventing the broad applicability of LnGMs. In this article, we derive variational Bayes algorithms for fast and scalable inference of LnGMs. The approximation leads to an LGM that downweights extreme events in the latent process, reducing their influence and leading to more robust inferences. It can be applied to a wide range of models, such as autoregressive processes for time series, simultaneous autoregressive models for areal data, and spatial Matérn models. To facilitate Bayesian inference, we introduce the ngvb package, where LGMs implemented in R-INLA can be easily extended to LnGMs by adding a single line of code. Supplementary materials for this article are available online. [ABSTRACT FROM AUTHOR]

We study the Bayesian multi-task variable selection problem, where the goal is to select activated variables for multiple related datasets simultaneously. We propose a new variational Bayes algorithm which generalizes and improves the recently developed "sum of single effects" model of Wang et al. Motivated by differential gene network analysis in biology, we further extend our method to joint structure learning of multiple directed acyclic graphical models, a problem known to be computationally highly challenging. We propose a novel order MCMC sampler where our multi-task variable selection algorithm is used to quickly evaluate the posterior probability of each ordering. Both simulation studies and real gene expression data analysis are conducted to show the efficiency of our method. Finally, we also prove a posterior consistency result for multi-task variable selection, which provides a theoretical guarantee for the proposed algorithms. for this article are available online. [ABSTRACT FROM AUTHOR]

AbstractMarkov chain Monte Carlo (MCMC) provides asymptotically consistent estimates of intractable posterior expectations as the number of iterations tends to infinity. However, in large data applications, MCMC can be computationally expensive per iteration. This has catalyzed interest in approximating MCMC in a manner that improves computational speed per iteration but does not produce asymptotically consistent estimates. In this article, we propose estimators based on couplings of Markov chains to assess the quality of such asymptotically biased sampling methods. The estimators give empirical upper bounds of the Wasserstein distance between the limiting distribution of the asymptotically biased sampling method and the original target distribution of interest. We establish theoretical guarantees for our upper bounds and show that our estimators can remain effective in high dimensions. We apply our quality measures to stochastic gradient MCMC, variational Bayes, and Laplace approximations for tall data and to approximate MCMC for Bayesian logistic regression in 4500 dimensions and Bayesian linear regression in 50,000 dimensions. [ABSTRACT FROM AUTHOR]

This paper presents a probabilistic model for predicting the dynamic modulus |E*| of asphalt concrete (AC). A Bayesian Neural Network (BNN) trained on a substantial dataset collected from various states was employed. This approach accounts for the inherent stochasticity in the data generation process (i.e. aleatoric uncertainty) and addresses epistemic or model uncertainty as well. The model successfully predicted dynamic moduli for unseen testing datasets. In practice, predicted moduli could be as accurate and effective as measured values, considering the model uncertainty and |E*| test variability. However, an elevated epistemic uncertainty at extremely low and high |E*| ranges would be expected due to relatively low data points. To enhance interpretability, Shapley Additive Explanations (SHAP) analysis was utilised, demonstrating adherence to physical laws. The primary factors influencing predicted moduli, as identified by the analysis, are temperature, frequency, voids in mineral aggregates, reclaimed asphalt pavement content, and binder low-performance grade. The BNN model was deployed in a web application, making it accessible for application. [ABSTRACT FROM AUTHOR]

The multinomial probit model is often used to analyze choice behavior. However, estimation with existing Markov chain Monte Carlo (MCMC) methods is computationally costly, which limits its applicability to large choice datasets. This article proposes a variational Bayes method that is accurate and fast, even when a large number of choice alternatives and observations are considered. Variational methods usually require an analytical expression for the unnormalized posterior density and an adequate choice of variational family. Both are challenging to specify in a multinomial probit, which has a posterior that requires identifying restrictions and is augmented with a large set of latent utilities. We employ a spherical transformation on the covariance matrix of the latent utilities to construct an unnormalized augmented posterior that identifies the parameters, and use the conditional posterior of the latent utilities as part of the variational family. The proposed method is faster than MCMC, and can be made scalable to both a large number of choice alternatives and a large number of observations. The accuracy and scalability of our method is illustrated in numerical experiments and real purchase data with one million observations. [ABSTRACT FROM AUTHOR]

Using theoretical and numerical results, we document the accuracy of commonly applied variational Bayes methods across a range of state space models. The results demonstrate that, in terms of accuracy on fixed parameters, there is a clear hierarchy in terms of the methods, with approaches that adequately approximate the states yielding superior accuracy over methods that do not. We also document numerically that over small out-of-sample evaluation periods the inferential discrepancies between the various methods often yield only small discrepancies in predictive accuracy. Nevertheless, in certain settings, and over a longer out-of-sample period, these predictive discrepancies can become meaningful. This finding indicates that the invariance of predictive results to inferential inaccuracy, which has been an oft-touted point made by practitioners seeking to justify the use of variational inference, is not ubiquitous and must be assessed on a case-by-case basis. for this article are available online. [ABSTRACT FROM AUTHOR]

Accurate estimation of the change in crime over time is a critical first step toward better understanding of public safety in large urban environments. Bayesian hierarchical modeling is a natural way to study spatial variation in urban crime dynamics at the neighborhood level, since it facilitates principled "sharing of information" between spatially adjacent neighborhoods. Typically, however, cities contain many physical and social boundaries that may manifest as spatial discontinuities in crime patterns. In this situation, standard prior choices often yield overly smooth parameter estimates, which can ultimately produce mis-calibrated forecasts. To prevent potential over-smoothing, we introduce a prior that partitions the set of neighborhoods into several clusters and encourages spatial smoothness within each cluster. In terms of model implementation, conventional stochastic search techniques are computationally prohibitive, as they must traverse a combinatorially vast space of partitions. We introduce an ensemble optimization procedure that simultaneously identifies several high probability partitions by solving one optimization problem using a new local search strategy. We then use the identified partitions to estimate crime trends in Philadelphia between 2006 and 2017. On simulated and real data, our proposed method demonstrates good estimation and partition selection performance. for this article are available online. [ABSTRACT FROM AUTHOR]

High-dimensional classification and feature selection tasks are ubiquitous with the recent advancement in data acquisition technology. In several application areas such as biology, genomics, and proteomics, the data are often functional in their nature and exhibit a degree of roughness and nonstationarity. These structures pose additional challenges to commonly used methods that rely mainly on a two-stage approach performing variable selection and classification separately. We propose in this work a novel Gaussian process discriminant analysis (GPDA) that combines these steps in a unified framework. Our model is a two-layer nonstationary Gaussian process coupled with an Ising prior to identify differentially-distributed locations. Scalable inference is achieved via developing a variational scheme that exploits advances in the use of sparse inverse covariance matrices. We demonstrate the performance of our methodology on simulated datasets and two proteomics datasets: breast cancer and SARS-CoV-2. Our approach distinguishes itself by offering explainability as well as uncertainty quantification in addition to low computational cost, which are crucial to increase trust and social acceptance of data-driven tools. for this article are available online. [ABSTRACT FROM AUTHOR]

Stochastic blockmodels have been widely proposed as a probabilistic random graph model for the analysis of networks data as well as for detecting community structure in these networks. In a number of real-world networks, not all ties among nodes have the same weight. Ties among networks nodes are often associated with weights that differentiate them in terms of their strength, intensity, or capacity. In this paper, we are interested in the case of co-citation networks, where the nodes are words and each edge joining a pair of words is weighted by the number of co-citation of these two words together in the same document. In this type of networks, the weight associated to each edge is an integer value bounded by the the whole number of documents in the considered corpus. Hence, we propose an extension of the stochastic blockmodels to deal with the case of a binomial distribution for the edge's weights. We provide an inference method through a variational expectation maximization algorithm to estimate the parameters in binomial stochastic blockmodels for weighted networks. To prove the validity of the method and to highlight its main features, we set some applications of the proposed approach by using some simulated data and then some real data sets. Stochastic blockmodels belong to latent classes models. Classes defines a node's clustering. We compare the clustering found through binomial stochastic blockmodels with the ones found fitting a stochastic blockmodel with Poisson distributed edges' weights. Inferred Poisson and binomial stochastic blockmodels mainly differs. Moreover, in our examples, the statistical error is lower for binomial stochastic blockmodels. [ABSTRACT FROM AUTHOR]

In this paper, we estimate the dynamic parameters of a time-varying coefficient model through radial kernel functions in the context of a longitudinal study. Our proposal is based on a linear combination of weighted kernel functions involving a bandwidth, centered around a given set of time points. In addition, we study different alternatives of estimation and inference including a Frequentist approach using weighted least squares along with bootstrap methods, and a Bayesian approach through both Markov chain Monte Carlo and variational methods. We compare the estimation strategies mention above with each other, and our radial kernel functions proposal with an expansion based on regression spline, by means of an extensive simulation study considering multiples scenarios in terms of sample size, number of repeated measurements, and subject-specific correlation. Our experiments show that the capabilities of our proposal based on radial kernel functions are indeed comparable with or even better than those obtained from regression splines. We illustrate our methodology by analyzing data from two AIDS clinical studies. [ABSTRACT FROM AUTHOR]

Mean-field variational inference, built on fully factorisations, can be efficiently solved; however, it ignores the dependencies between latent variables, resulting in lower performance. To address this, the copula variational inference (CVI) method is proposed by using the well-established copulas to effectively capture posterior dependencies, leading to better approximations. However, it suffers from a computational issue, where the optimisation for big models with massive latent variables is quite time-consuming. This is mainly caused by the expensive sampling when forming noisy Monte Carlo gradients in CVI. For CVI speedup, in this paper we propose a novel fast CVI (abbr. FCVI). In FCVI, we derive the gradient of CVI objective by an expectation of the mean-field factorisation. Therefore, we can achieve a much efficient sampling from the D -dimensional mean-field factorisation, enabling to reduce the sampling complexity from O (D 2) to O (D). To evaluate FCVI, we compare it against baseline methods on modelling performance and runtime. Experimental results demonstrate that FCVI is on a par with CVI, but runs much faster. [ABSTRACT FROM AUTHOR]

With the advance of next generation sequencing technologies, researchers now routinely obtain a collection of microbial sequences with complex phylogenetic relationships. It is often of interest to analyze the association between certain environmental factors and characteristics of the microbial collection. Though methods have been developed to test for association between the microbial composition with environmental factors as well as between coevolving traits, a flexible model that can provide a comprehensive picture of the relationship between microbial community characteristics and environmental variables will be tremendously beneficial. We developed a Bayesian approach for association analysis while incorporating the phylogenetic structure to account for the dependence between observations. To overcome the computational difficulty related to the phylogenetic tree, a variational algorithm was developed to evaluate the posterior distribution. As the posterior distribution can be readily obtained for parameters of interest and any derived variables, the association relationship can be examined comprehensively. With two application examples, we demonstrated that the Bayesian approach can uncover nuanced details of the microbial assemblage with regard to the environmental factor. The proposed Bayesian approach and variational algorithm can be extended for other problems involving dependence over tree-like structures. [ABSTRACT FROM AUTHOR]

Network estimation and variable selection have been extensively studied in the statistical literature, but only recently have those two challenges been addressed simultaneously. In this article, we seek to develop a novel method to simultaneously estimate network interactions and associations to relevant covariates for count data, and specifically for compositional data, which have a fixed sum constraint. We use a hierarchical Bayesian model with latent layers and employ spike-and-slab priors for both edge and covariate selection. For posterior inference, we develop a novel variational inference scheme with an expectation–maximization step, to enable efficient estimation. Through simulation studies, we demonstrate that the proposed model outperforms existing methods in its accuracy of network recovery. We show the practical utility of our model via an application to microbiome data. The human microbiome has been shown to contribute too many of the functions of the human body, and also to be linked with a number of diseases. In our application, we seek to better understand the interaction between microbes and relevant covariates, as well as the interaction of microbes with each other. We call our algorithm simultaneous inference for networks and covariates and provide a Python implementation, which is available online. [ABSTRACT FROM AUTHOR]

The coevolution between human and bacteria colonizing the human body has profound implications for heath and development, with a growing body of evidence linking the altered microbiome composition with a wide array of disease states. Yet dimension reduction and visualization analysis of microbiome data are still in their infancy and many challenges exist. In this article, we introduce a general framework, zero-inflated probabilistic principal component analysis (ZIPPCA), for dimension reduction and data ordination of multivariate abundance data, and propose an efficient variational approximation method for estimation, inference, and prediction. Extensive simulations show that the proposed method outperforms algorithm-based methods and compares favorably with existing model-based methods. We further apply our method to a gut microbiome dataset for visualization analysis of community composition across age and geography. The method is implemented in R and available at . [ABSTRACT FROM AUTHOR]

Deep learning has emerged as a powerful tool to model complicated relationships between inputs and outputs in various fields including degradation modeling and prognostics. Existing deep learning-based prognostic approaches are often used in a black-box manner and provide only point estimations of remaining useful life. However, accurate interval estimations of the remaining useful life are crucial to understand the stochastic nature of degradation processes and perform reliable risk analysis and maintenance decision making. This study proposes a novel Bayesian deep learning framework that incorporates general characteristics of degradation processes and provides the interval estimations of remaining useful life. The proposed method enjoys several unique advantages: (i) providing a general approach by not assuming any particular type of degradation processes nor the availability of domain-specific prior knowledge such as a failure threshold; (ii) offering the interval estimations of the remaining useful life; (iii) systematically modeling two types of uncertainties embedded in prognostics; and (iv) exhibiting great prognostic performance and wide applicability to complex systems that may involve multiple sensor signals, multiple failure modes, and multiple operational conditions. Numerical studies demonstrate improved prognostic performance and practicality of the proposed method over benchmark approaches. Additional numerical results including the analysis of sensitivity and computational costs are given in the online supplemental materials. [ABSTRACT FROM AUTHOR]

Aiming at complicated remote sensing image registration problem, we proposed a multi-constraint registration algorithm based on variational inference. Gaussian mixture model is used to construct the entire Bayesian network, which facilitates the estimation of heteroscedasticity noise in mapping function regression, and distribution of outliers in scene point density estimation. We use a variational parameter proxy to approximate complex posterior distribution. In addition, while maintaining the stability of global structure, similarity of local structure is restricted. In order to limit the cost of image conversion, we added constraints on spatial curvature for image. Accuracy and smoothness of the spatial transformation can be maintained by minimizing the curvature in horizontal and vertical directions. Experimental results show that compared with other point set matching algorithms, this method can achieve better results in robustness and matching accuracy. [ABSTRACT FROM AUTHOR]

Haptic guidance is a powerful technique to combine the strengths of humans and autonomous systems for teleoperation. The autonomous system can provide haptic cues to enable the operator to perform precise movements; the operator can interfere with the plan of the autonomous system leveraging his/her superior cognitive capabilities. However, providing haptic cues such that the individual strengths are not impaired is challenging because low forces provide little guidance, whereas strong forces can hinder the operator in realizing his/her plan. Based on variational inference, we learn a Gaussian mixture model (GMM) over trajectories to accomplish a given task. The learned GMM is used to construct a potential field which determines the haptic cues. The potential field smoothly changes during teleoperation based on our updated belief over the plans and their respective phases. Furthermore, new plans are learned online when the operator does not follow any of the proposed plans or after changes in the environment. User studies confirm that our framework helps users perform teleoperation tasks more accurately than without haptic cues and, in some cases, faster. Moreover, we demonstrate the use of our framework to help a subject teleoperate a 7 DoF manipulator in a pick-and-place task. [ABSTRACT FROM AUTHOR]

Water pollution is a major global environmental problem, and it poses a great environmental risk to public health and biological diversity. This work is motivated by assessing the potential environmental threat of coal mining through increased sulfate concentrations in river networks, which do not belong to any simple parametric distribution. However, existing network models mainly focus on binary or discrete networks and weighted networks with known parametric weight distributions. We propose a principled nonparametric weighted network model based on exponential-family random graph models and local likelihood estimation, and study its model-based clustering with application to large-scale water pollution network analysis. We do not require any parametric distribution assumption on network weights. The proposed method greatly extends the methodology and applicability of statistical network models. Furthermore, it is scalable to large and complex networks in large-scale environmental studies. The power of our proposed methods is demonstrated in simulation studies and a real application to sulfate pollution network analysis in Ohio watershed located in Pennsylvania, United States. [ABSTRACT FROM AUTHOR]

This article deals with nonobserved dyads during the sampling of a network and consecutive issues in the inference of the stochastic block model (SBM). We review sampling designs and recover missing at random (MAR) and not missing at random (NMAR) conditions for the SBM. We introduce variants of the variational EM algorithm for inferring the SBM under various sampling designs (MAR and NMAR) all available as an R package. Model selection criteria based on integrated classification likelihood are derived for selecting both the number of blocks and the sampling design. We investigate the accuracy and the range of applicability of these algorithms with simulations. We explore two real-world networks from ethnology (seed circulation network) and biology (protein–protein interaction network), where the interpretations considerably depend on the sampling designs considered. for this article are available online. [ABSTRACT FROM AUTHOR]

In this article, we consider a Bayesian bi-level variable selection problem in high-dimensional regressions. In many practical situations, it is natural to assign group membership to each predictor. Examples include that genetic variants can be grouped at the gene level and a covariate from different tasks naturally forms a group. Thus, it is of interest to select important groups as well as important members from those groups. The existing Markov chain Monte Carlo methods are often computationally intensive and not scalable to large datasets. To address this problem, we consider variational inference for bi-level variable selection. In contrast to the commonly used mean-field approximation, we propose a hierarchical factorization to approximate the posterior distribution, by using the structure of bi-level variable selection. Moreover, we develop a computationally efficient and fully parallelizable algorithm based on this variational approximation. We further extend the developed method to model datasets from multitask learning. The comprehensive numerical results from both simulation studies and real data analysis demonstrate the advantages of BIVAS for variable selection, parameter estimation, and computational efficiency over existing methods. The method is implemented in R package "bivas" available at . for this article are available online. [ABSTRACT FROM AUTHOR]

In this article, we employ the variational Bayesian method to study the parameter estimation problems of linear regression model, wherein some regressors are of Gaussian distribution with nonzero prior means. We obtain an analytical expression of the posterior parameter distribution, and then propose an iterative algorithm for the model. Simulations are carried out to test the performance of the proposed algorithm, and the simulation results confirm both the effectiveness and the reliability of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Markov chain Monte Carlo approaches have been widely used for Bayesian inference. The drawback of these methods is that they can be computationally prohibitive especially when complex models are analyzed. In such cases, variational methods may provide an efficient and attractive alternative. However, the variational methods reported to date are applicable to relatively simple models and most are based on a factorized approximation to the posterior distribution. Here, we propose a variational approach that is capable of handling models that consist of a system of differential-algebraic equations and whose posterior approximation can be represented by a multivariate distribution. Under the proposed approach, the solution of the variational inference problem is decomposed into three steps: a maximum a posteriori optimization, which is facilitated by using an orthogonal collocation approach, a preprocessing step, which is based on the estimation of the eigenvectors of the posterior covariance matrix, and an expected propagation optimization problem. To tackle multivariate integration, we employ quadratures derived from the Smolyak rule (sparse grids). Examples are reported to elucidate the advantages and limitations of the proposed methodology. The results are compared to the solutions obtained from a Markov chain Monte Carlo approach. It is demonstrated that significant computational savings can be gained using the proposed approach. This article has supplementary material online. [ABSTRACT FROM AUTHOR]

Variational and variational Bayes techniques are popular approaches for statistical inference of complex models but their theoretical properties are still not well known. Because of both unobserved variables and intricate dependency structures, mixture models for random graphs constitute a good case study. We first present four different variational estimates for the parameters of these models. We then compare their accuracy through simulation studies and show that the variational Bayes estimates seem the most accurate for moderate graph size. We finally re-analyse the regulatory network of Escherichia coli with this approach. [ABSTRACT FROM AUTHOR]