Search

• Streaming data are characterised as evolution of Stochastic Differential Equations. • Gaussian process regression used for nonparametric estimation of SDE coefficients. • Stochastic process theory applied to automatically compare SDE models. • Bootstrapping methods allow for control of the false alarm rate of detector. • Evaluation on datasets demonstrates detector's high discriminative power. This paper characterises streaming data as the evolution of a stochastic differential equation, with the aim of extracting information that can be used to detect anomalies in the stream. Gaussian process regression provides a flexible approach to approximating components of the stochastic differential equation, allowing for complex modelling of underlying data generation dynamics. The proposed algorithm displays superior discriminative power over different time-series anomaly detection methods for both synthetic and NYC taxi datasets, whilst the introduced bootstrapping method for setting the detection threshold provides control over the false alarm rate of the anomaly detector. [ABSTRACT FROM AUTHOR]

Aeroengine performance is determined by temperature and pressure profiles along various axial stations within an engine. Given limited sensor measurements, we require a statistically principled approach for inferring these profiles. In this paper we detail a Bayesian methodology for interpolating the spatial temperature or pressure profile at axial stations within an aeroengine. The profile at any given axial station is represented as a spatial Gaussian random field on an annulus, with circumferential variations modelled using a Fourier basis and radial variations modelled with a squared exponential kernel. This Gaussian random field is extended to ingest data from multiple axial measurement planes, with the aim of transferring information across the planes. To facilitate this type of transfer learning, a novel planar covariance kernel is proposed. In the scenario where frequencies comprising the temperature field are unknown, we utilise a sparsity-promoting prior on the frequencies to encourage sparse representations. This easily extends to cases with multiple engine planes whilst accommodating frequency variations between the planes. The main quantity of interest, the spatial area average is readily obtained in closed form. We term this the Bayesian area average and demonstrate how this metric offers far more representative averages than a sector area average--a widely used area averaging approach. Furthermore, the Bayesian area average naturally decomposes the posterior uncertainty into terms characterising insufficient sampling and sensor measurement error respectively. This too provides a significant improvement over prior standard deviation based uncertainty breakdowns. [ABSTRACT FROM AUTHOR]

Recent research has shown the potential utility of deep Gaussian processes. These deep structures are probability distributions, designed through hierarchical construction, which are conditionally Gaussian. In this paper, the current published body of work is placed in a common framework and, through recursion, several classes of deep Gaussian processes are defined. The resulting samples generated from a deep Gaussian process have a Markovian structure with respect to the depth parameter, and the effective depth of the resulting process is interpreted in terms of the ergodicity, or non-ergodicity, of the resulting Markov chain. For the classes of deep Gaussian processes introduced, we provide results concerning their ergodicity and hence their effective depth. We also demonstrate how these processes may be used for inference; in particular we show how a Metropolis-within-Gibbs construction across the levels of the hierarchy can be used to derive sampling tools which are robust to the level of resolution used to represent the functions on a computer. For illustration, we consider the effect of ergodicity in some simple numerical examples. [ABSTRACT FROM AUTHOR]

Bat echolocation call identification methods are important in developing efficient cost-effective methods for large-scale bioacoustic surveys for global biodiversity monitoring and conservation planning. Such methods need to provide interpretable probabilistic predictions of species since they will be applied across many different taxa in a diverse set of applications and environments. We develop such a method using a multinomial probit likelihood with independent Gaussian process priors and study its feasibility on a data set from an on-going study of 21 species, five families and 1800 bat echolocation calls collected from Mexico, a hotspot of bat biodiversity. We propose an efficient approximate inference scheme based on the expectation propagation algorithm and observe that the overall methodology significantly improves on currently adopted approaches to bat call classification by providing an approach which can be easily generalized across different species and call types and is fully probabilistic. Implementation of this method has the potential to provide robust species identification tools for biodiversity acoustic bat monitoring programmes across a range of taxa and spatial scales. [ABSTRACT FROM AUTHOR]

We propose a determinant-free approach for simulation-based Bayesian inference in high-dimensional Gaussian models. We introduce auxiliary variables with covariance equal to the inverse covariance of the model. The joint probability of the auxiliary model can be computed without evaluating determinants, which are often hard to compute in high dimensions. We develop a Markov chain Monte Carlo sampling scheme for the auxiliary model that requires no more than the application of inverse-matrix-square-roots and the solution of linear systems. These operations can be performed at large scales with rational approximations. We provide an empirical study on both synthetic and real-world data for sparse Gaussian processes and for large-scale Gaussian Markov random fields. Copyright © 2017 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

The main challenges that arise when adopting Gaussian process priors in probabilistic modeling are how to carry out exact Bayesian inference and how to account for uncertainty on model parameters when making model-based predictions on out-of-sample data. Using probit regression as an illustrative working example, this paper presents a general and effective methodology based on the pseudo-marginal approach to Markov chain Monte Carlo that efficiently addresses both of these issues. The results presented in this paper show improvements over existing sampling methods to simulate from the posterior distribution over the parameters defining the covariance function of the Gaussian Process prior. This is particularly important as it offers a powerful tool to carry out full Bayesian inference of Gaussian Process based hierarchic statistical models in general. The results also demonstrate that Monte Carlo based integration of all model parameters is actually feasible in this class of models providing a superior quantification of uncertainty in predictions. Extensive comparisons with respect to state-of-the-art probabilistic classifiers confirm this assertion. [ABSTRACT FROM AUTHOR]

The paper proposes Metropolis adjusted Langevin and Hamiltonian Monte Carlo sampling methods defined on the Riemann manifold to resolve the shortcomings of existing Monte Carlo algorithms when sampling from target densities that may be high dimensional and exhibit strong correlations. The methods provide fully automated adaptation mechanisms that circumvent the costly pilot runs that are required to tune proposal densities for Metropolis-Hastings or indeed Hamiltonian Monte Carlo and Metropolis adjusted Langevin algorithms. This allows for highly efficient sampling even in very high dimensions where different scalings may be required for the transient and stationary phases of the Markov chain. The methodology proposed exploits the Riemann geometry of the parameter space of statistical models and thus automatically adapts to the local structure when simulating paths across this manifold, providing highly efficient convergence and exploration of the target density. The performance of these Riemann manifold Monte Carlo methods is rigorously assessed by performing inference on logistic regression models, log-Gaussian Cox point processes, stochastic volatility models and Bayesian estimation of dynamic systems described by non-linear differential equations. Substantial improvements in the time-normalized effective sample size are reported when compared with alternative sampling approaches. MATLAB code that is available from allows replication of all the results reported. [ABSTRACT FROM AUTHOR]

Datasets that are subjectively labeled by a number of experts are becoming more common in tasks such as biological text annotation where class definitions are necessarily somewhat subjective. Standard classification and regression models are not suited to multiple labels and typically a pre-processing step (normally assigning the majority class) is performed. We propose Bayesian models for classification and ordinal regression that naturally incorporate multiple expert opinions in defining predictive distributions. The models make use of Gaussian process priors, resulting in great flexibility and particular suitability to text based problems where the number of covariates can be far greater than the number of data instances. We show that using all labels rather than just the majority improves performance on a recent biological dataset. [ABSTRACT FROM AUTHOR]

It is well known in the statistics literature that augmenting binary and polychotomous response models with gaussian latent variables enables exact Bayesian analysis via Gibbs sampling from the parameter posterior. By adopting such a data augmentation strategy, dispensing with priors over regression coefficients in favor of gaussian process (GP) priors over functions, and employing variational approximations to the full posterior, we obtain efficient computational methods for GP classification in the multiclass setting.1 The model augmentation with additional latent variables ensures full a posteriori class coupling while retaining the simple a priori independent GP covariance structure from which sparse approximations, such as multiclass informative vector machines (IVM), emerge in a natural and straightforward manner. This is the first time that a fully variational Bayesian treatment for multiclass GP classification has been developed without having to resort to additional explicit approximations to the nongaussian likelihood term. Empirical comparisons with exact analysis use Markov Chain Monte Carlo (MCMC) and Laplace approximations illustrate the utility of the variational approximation as a computationally economic alternative to full MCMC and it is shown to be more accurate than the Laplace approximation. [ABSTRACT FROM AUTHOR]

This article develops an extended independent component analysis algorithm for mixtures of arbitrary subgaussian and supergaussian sources. The gaussian mixture model of Pearson is employed in deriving a closedform generic score function for strictly subgaussian sources. This is combined with the score function for a unimodal supergaussian density to provide a computationally simple yet powerful algorithm for performing independent component analysis on arbitrary mixtures of nongaussian sources. [ABSTRACT FROM AUTHOR]

The increased availability of observation data from engineering systems in operation poses the question of how to incorporate this data into finite element models. To this end, we propose a novel statistical construction of the finite element method that provides the means of synthesising measurement data and finite element models. The Bayesian statistical framework is adopted to treat all the uncertainties present in the data, the mathematical model and its finite element discretisation. From the outset, we postulate a statistical generating model which additively decomposes data into a finite element, a model misspecification and a noise component. Each of the components may be uncertain and is considered as a random variable with a respective prior probability density. The prior of the finite element component is given by a conventional stochastic forward problem. The prior probabilities of the model misspecification and measurement noise, without loss of generality, are assumed to have a zero-mean and a known covariance structure. Our proposed statistical model is hierarchical in the sense that each of the three random components may depend on one or more non-observable random hyperparameters with their own corresponding probability densities. We use Bayes rule to infer the posterior densities of the three random components and the hyperparameters from their known prior densities and a data dependent likelihood function. Because of the hierarchical structure of our statistical model, Bayes rule is applied on three different levels in turn. On level one, we determine the posterior densities of the finite element component and the true system response using the prior finite element density given by the forward problem and the data likelihood. In this step, approximating the prior finite element density with a multivariate Gaussian distribution allows us to obtain a closed-form expression for the posterior. On the next level, we infer the hyperparameter posterior densities from their respective priors and the marginal likelihood of the first inference problem. These posteriors are sampled numerically using the Markov chain Monte Carlo (MCMC) method. Finally, on level three we use Bayes rule to choose the most suitable finite element model in light of the observed data by computing the respective model posteriors. We demonstrate the application and versatility of statFEM with one and two-dimensional examples. • Statistical FE approach for synthesis of observation data and FE models is proposed. • A data-generating process combining FE model, model mismatch and data is assumed. • All random variables and hyperparameters are learned using Bayesian inference. • Closed-form expressions for the posterior FE and true system probabilities are derived. • Convergence of the posterior probabilities with increasing data is demonstrated. [ABSTRACT FROM AUTHOR]

Gaussian processes are ubiquitous in machine learning, statistics, and applied mathematics. They provide a flexible modelling framework for approximating functions, whilst simultaneously quantifying uncertainty. However, this is only true when the model is well-specified, which is often not the case in practice. In this paper, we study the properties of Gaussian process means when the smoothness of the model and the likelihood function are misspecified. In this setting, an important theoretical question of practical relevance is how accurate the Gaussian process approximations will be given the chosen model and the extent of the misspecification. The answer to this problem is particularly useful since it can inform our choice of model and experimental design. In particular, we describe how the experimental design and choice of kernel and kernel hyperparameters can be adapted to alleviate model misspecification. [ABSTRACT FROM AUTHOR]

Summary: We present a new C implementation of an advanced Markov chain Monte Carlo (MCMC) method for the sampling of ordinary differential equation (ode) model parameters. The software mcmc_clib uses the simplified manifold Metropolis-adjusted Langevin algorithm (SMMALA), which is locally adaptive; it uses the parameter manifold’s geometry (the Fisher information) to make efficient moves. This adaptation does not diminish with MC length, which is highly advantageous compared with adaptive Metropolis techniques when the parameters have large correlations and/or posteriors substantially differ from multivariate Gaussians. The software is standalone (not a toolbox), though dependencies include the GNU scientific library and sundials libraries for ode integration and sensitivity analysis.Availability and implementation: The source code and binary files are freely available for download at http://a-kramer.github.io/mcmc_clib/. This also includes example files and data. A detailed documentation, an example model and user manual are provided with the software.Contact: andrei.kramer@ist.uni-stuttgart.de [ABSTRACT FROM AUTHOR]

At the heart of structural engineering research is the use of data obtained from physical structures such as bridges, viaducts and buildings. These data can represent how the structure responds to various stimuli over time when in operation. Many models have been proposed in literature to represent such data, such as linear statistical models. Based upon these models, the health of the structure is reasoned about, e.g. through damage indices, changes in likelihood and statistical parameter estimates. On the other hand, physics-based models are typically used when designing structures to predict how the structure will respond to operational stimuli. These models represent how the structure responds to stimuli under idealised conditions. What remains unclear in the literature is how to combine the observed data with information from the idealised physics-based model into a model that describes the responses of the operational structure. This paper introduces a new approach which fuses together observed data from a physical structure during operation and information from a mathematical model. The observed data are combined with data simulated from the physics-based model using a multi-output Gaussian process formulation. The novelty of this method is how the information from observed data and the physics-based model is balanced to obtain a representative model of the structures response to stimuli. We present our method using data obtained from a fibre-optic sensor network installed on experimental railway sleepers. The curvature of the sleeper at sensor and also non-sensor locations is modelled, guided by the mathematical representation. We discuss how this approach can be used to reason about changes in the structures behaviour over time using simulations and experimental data. The results show that the methodology can accurately detect such changes. They also indicate that the methodology can infer information about changes in the parameters within the physics-based model, including those governing components of the structure not measured directly by sensors such as the ballast foundation. • Probabilistic models can be used to balance information obtained from physics-based models and observed data. • Balancing is implemented by maximizing the predictive performance of observed test data-points. • Physics-based models can improve the inference of the structural response in areas of the domain which are unmeasured. • The proposed information synthesis can infer model parameters that govern structural components that cannot be measured. • Experimental application of the proposed methodology, detecting underlying changes in model parameters resulting from damage. [ABSTRACT FROM AUTHOR]

Significant funds have been allocated to maintain road networks each year in developed countries. Performance prediction is crucial for pavement management systems to adjust working plans and budget allocation. As a Bayesian nonparametric method, Gaussian process regression (GPR) is powerful in predicting nonlinear time series and quantifying uncertainty. However, it remains computationally intensive and fails to adapt to the time-varying characteristics. To address such issues, a dynamic GPR model is proposed for probabilistic prediction of the International Roughness Index (IRI) for flexible pavements. A moving window strategy is developed to substantially shrink the size of training data, which effectively alleviates computational cost and thus leads to a dynamic GPR. A genetic algorithm is then adopted to determine the optimal window size by considering the trade-off between computational efficiency and accuracy. A dataset acquired from Long-Term Pavement Performance (LTPP) is used to demonstrate the feasibility of the dynamic GPR. Its performance is compared to traditional GPR as well as dynamic and static Bayesian linear regression (BLR) models. The comparison results indicate that the proposed dynamic GPR can increase the accuracy by 0.86, 1.52, and 2.27 times for dynamic BLR, static GPR, and static BLR, respectively. It exhibits the best results in terms of accuracy and uncertainty metrics due to its nonlinear modelling and time-varying ability. • A dynamic Gaussian process regression (GPR) model is proposed for probabilistic prediction of pavement conditions. • A moving window strategy is developed to effectively alleviate the computational cost. • A genetic algorithm is adopted to determine the optimal window size. • The traditional GPR as well as dynamic and static Bayesian linear regression models are employed for comparison. [ABSTRACT FROM AUTHOR]

Abstract: Classifying electroencephalography (EEG) signals is an important step for proceeding EEG-based brain computer interfaces (BCI). Currently, kernel based methods such as the support vector machine (SVM) are considered the state-of-the-art methods for this problem. In this paper, we apply Gaussian process (GP) classification to binary discrimination of motor imagery EEG data. Compared with the SVM, GP based methods naturally provide probability outputs for identifying a trusted prediction which can be used for post-processing in a BCI. Experimental results show that the classification methods based on a GP perform similarly to kernel logistic regression and probabilistic SVM in terms of predictive likelihood, but outperform SVM and K-nearest neighbor (KNN) in terms of 0–1 loss class prediction error. [Copyright &y& Elsevier]