Search

• Streaming data are characterised as evolution of Stochastic Differential Equations. • Gaussian process regression used for nonparametric estimation of SDE coefficients. • Stochastic process theory applied to automatically compare SDE models. • Bootstrapping methods allow for control of the false alarm rate of detector. • Evaluation on datasets demonstrates detector's high discriminative power. This paper characterises streaming data as the evolution of a stochastic differential equation, with the aim of extracting information that can be used to detect anomalies in the stream. Gaussian process regression provides a flexible approach to approximating components of the stochastic differential equation, allowing for complex modelling of underlying data generation dynamics. The proposed algorithm displays superior discriminative power over different time-series anomaly detection methods for both synthetic and NYC taxi datasets, whilst the introduced bootstrapping method for setting the detection threshold provides control over the false alarm rate of the anomaly detector. [ABSTRACT FROM AUTHOR]

A key challenge in agent‐based mobility simulations is the synthesis of individual agent socioeconomic profiles. Such profiles include locations of agent activities, which dictate the quality of the simulated travel patterns. These locations are typically represented in origin‐destination matrices that are sampled using coarse travel surveys. This is because fine‐grained trip profiles are scarce and fragmented due to privacy and cost reasons. The discrepancy between data and sampling resolutions renders agent traits nonidentifiable due to the combinatorial space of data‐consistent individual attributes. This problem is pertinent to any agent‐based inference setting where the latent state is discrete. Existing approaches have used continuous relaxations of the underlying location assignments and subsequent ad hoc discretisation thereof. We propose a framework to efficiently navigate this space offering improved reconstruction and coverage as well as linear‐time sampling of the ground truth origin‐destination table. This allows us to avoid factorially growing rejection rates and poor summary statistic consistency inherent in discrete choice modelling. We achieve this by introducing joint sampling schemes for the continuous intensity and discrete table of agent trips, as well as Markov bases that can efficiently traverse this combinatorial space subject to summary statistic constraints. Our framework's benefits are demonstrated in multiple controlled experiments and a large‐scale application to agent work trip reconstruction in Cambridge, UK. [ABSTRACT FROM AUTHOR]

Recent research has shown the potential utility of deep Gaussian processes. These deep structures are probability distributions, designed through hierarchical construction, which are conditionally Gaussian. In this paper, the current published body of work is placed in a common framework and, through recursion, several classes of deep Gaussian processes are defined. The resulting samples generated from a deep Gaussian process have a Markovian structure with respect to the depth parameter, and the effective depth of the resulting process is interpreted in terms of the ergodicity, or non-ergodicity, of the resulting Markov chain. For the classes of deep Gaussian processes introduced, we provide results concerning their ergodicity and hence their effective depth. We also demonstrate how these processes may be used for inference; in particular we show how a Metropolis-within-Gibbs construction across the levels of the hierarchy can be used to derive sampling tools which are robust to the level of resolution used to represent the functions on a computer. For illustration, we consider the effect of ergodicity in some simple numerical examples. [ABSTRACT FROM AUTHOR]

Bayesian inverse problems often involve sampling posterior distributions on infinite-dimensional function spaces. Traditional Markov chain Monte Carlo (MCMC) algorithms are characterized by deteriorating mixing times upon mesh-refinement, when the finite-dimensional approximations become more accurate. Such methods are typically forced to reduce step-sizes as the discretization gets finer, and thus are expensive as a function of dimension. Recently, a new class of MCMC methods with mesh-independent convergence times has emerged. However, few of them take into account the geometry of the posterior informed by the data. At the same time, recently developed geometric MCMC algorithms have been found to be powerful in exploring complicated distributions that deviate significantly from elliptic Gaussian laws, but are in general computationally intractable for models defined in infinite dimensions. In this work, we combine geometric methods on a finite-dimensional subspace with mesh-independent infinite-dimensional approaches. Our objective is to speed up MCMC mixing times, without significantly increasing the computational cost per step (for instance, in comparison with the vanilla preconditioned Crank–Nicolson (pCN) method). This is achieved by using ideas from geometric MCMC to probe the complex structure of an intrinsic finite-dimensional subspace where most data information concentrates, while retaining robust mixing times as the dimension grows by using pCN-like methods in the complementary subspace. The resulting algorithms are demonstrated in the context of three challenging inverse problems arising in subsurface flow, heat conduction and incompressible flow control. The algorithms exhibit up to two orders of magnitude improvement in sampling efficiency when compared with the pCN method. [ABSTRACT FROM AUTHOR]

Failure assessment diagrams are an integral component of asset integrity management in a variety of industrial sectors. They allow for the assessment of the significance of cracks in structures, between the domains of brittle fracture and plastic collapse. Numerical modelling, or empirically determined assessment lines across this continuum, are the basis of the guidance in current industrial standards. However, as the requirement of such assessments progresses from demonstrating safety to optimising resilience and resource allocation, it will become necessary to compute more informative (probabilistic) estimates, which are compatible with decision analysis. In this paper, Bayesian regression models are proposed as a suitable method of quantifying (aleatory and epistemic) uncertainty in the limit state on a failure assessment diagram. The data considered in this study consists of laboratory tests completed on wide plate fracture specimens. This work is intended to address the inconsistencies in current editions of industrial standards, and limitations (regarding flexibility and application) of existing scientific literature on the topic. Potential applications are discussed, including the use of fracture mechanics in meaningful reliability analysis, quantitative risk management, and optimising experimental design for future material tests. • A probabilistic failure assessment diagram is compared to methods used in industry. • The proposed gaussian process model is flexible, non-linear and probabilistic. • It can therefore approximate the behaviour between brittle and plastic domains. • Applications of reliability analysis and experimental design are demonstrated. [ABSTRACT FROM AUTHOR]

The main challenges that arise when adopting Gaussian process priors in probabilistic modeling are how to carry out exact Bayesian inference and how to account for uncertainty on model parameters when making model-based predictions on out-of-sample data. Using probit regression as an illustrative working example, this paper presents a general and effective methodology based on the pseudo-marginal approach to Markov chain Monte Carlo that efficiently addresses both of these issues. The results presented in this paper show improvements over existing sampling methods to simulate from the posterior distribution over the parameters defining the covariance function of the Gaussian Process prior. This is particularly important as it offers a powerful tool to carry out full Bayesian inference of Gaussian Process based hierarchic statistical models in general. The results also demonstrate that Monte Carlo based integration of all model parameters is actually feasible in this class of models providing a superior quantification of uncertainty in predictions. Extensive comparisons with respect to state-of-the-art probabilistic classifiers confirm this assertion. [ABSTRACT FROM AUTHOR]

ABSTRACT Markov chain Monte Carlo methods explicitly defined on the manifold of probability distributions have recently been established. These methods are constructed from diffusions across the manifold and the solution of the equations describing geodesic flows in the Hamilton-Jacobi representation. This paper takes the differential geometric basis of Markov chain Monte Carlo further by considering methods to simulate from probability distributions that themselves are defined on a manifold, with common examples being classes of distributions describing directional statistics. Proposal mechanisms are developed based on the geodesic flows over the manifolds of support for the distributions, and illustrative examples are provided for the hypersphere and Stiefel manifold of orthonormal matrices. [ABSTRACT FROM AUTHOR]

This review examines kernel methods for online learning, in particular, multiclass classification. We examine margin-based approaches, stemming from Rosenblatt's original perceptron algorithm, as well as nonparametric probabilistic approaches that are based on the popular gaussian process framework. We also examine approaches to online learning that use combinations of kernels--online multiple kernel learning. We present empirical validation of a wide range of methods on a protein fold recognition data set, where different biological feature types are available, and two object recognition data sets, Caltech101 and Caltech256, where multiple feature spaces are available in terms of different image feature extraction methods. [ABSTRACT FROM AUTHOR]

This letter considers how a number of modern Markov chain Monte Carlo (MCMC)methods can be applied for parameter estimation and inference in state-space models with point process observations. We quantified the efficiencies of these MCMC methods on synthetic data, and our results suggest that the Reimannian manifold Hamiltonian Monte Carlo method offers the best performance. We further compared such a method with a previously tested variational Bayes method on two experimental data sets. Results indicate similar performance on the large data sets and superior performance on small ones. The work offers an extensive suite of MCMC algorithms evaluated on an important class of models for physiological signal analysis. [ABSTRACT FROM AUTHOR]

This letter considers howa number ofmodernMarkov chainMonteCarlo (MCMC)methods can be applied for parameter estimation and inference in state-space models with point process observations.We quantified the efficiencies of these MCMC methods on synthetic data, and our results suggest that the ReimannianmanifoldHamiltonianMonte Carlomethod offers the best performance. We further compared such a method with a previously tested variational Bayes method on two experimental data sets. Results indicate similar performance on the large data sets and superior performance on small ones. The work offers an extensive suite of MCMC algorithms evaluated on an important class of models for physiological signal analysis. [ABSTRACT FROM AUTHOR]

Abstract: In this paper we offer a variational Bayes approximation to the multinomial probit model for basis expansion and kernel combination. Our model is well-founded within a hierarchical Bayesian framework and is able to instructively combine available sources of information for multinomial classification. The proposed framework enables informative integration of possibly heterogeneous sources in a multitude of ways, from the simple summation of feature expansions to weighted product of kernels, and it is shown to match and in certain cases outperform the well-known ensemble learning approaches of combining individual classifiers. At the same time the approximation reduces considerably the CPU time and resources required with respect to both the ensemble learning methods and the full Markov chain Monte Carlo, Metropolis–Hastings within Gibbs solution of our model. We present our proposed framework together with extensive experimental studies on synthetic and benchmark datasets and also for the first time report a comparison between summation and product of individual kernels as possible different methods for constructing the composite kernel matrix. [Copyright &y& Elsevier]

Abstract: In this paper, we describe a Bayesian classification method that informatively combines diverse sources of information and multiple feature spaces for multiclass problems. The proposed method is based on recent advances in kernel approaches where the integration of multiple object descriptors, or feature spaces, is achieved via kernel combination. Each kernel constructs a similarity metric between objects in a particular feature space and then having a common metric across modalities an overall combination can be constructed. We follow a hierarchical Bayesian approach, which introduces prior distributions over random variables and we construct a Gibbs sampling Markov chain Monte Carlo (MCMC) solution which is naturally derived from the employed multinomial probit likelihood. The methodology is the basis for possible deterministic approximations such as variational or maximum-a-posteriori estimators, and it is compared against the well-known classifier combination methods on the classification of handwritten numerals. The results of the proposed method show a significant improvement over the best individual classifier and match the performance of the best multiple classifier combination, whilst reducing the computational requirements of combining classifiers and offering additional information on the significance of the contributing sources. [Copyright &y& Elsevier]

Abstract: Nonlinear dynamic systems such as biochemical pathways can be represented in abstract form using a number of modelling formalisms. In particular differential equations provide a highly expressive mathematical framework with which to model dynamic systems, and a very natural way to model the dynamics of a biochemical pathway in a deterministic manner is through the use of nonlinear ordinary or time delay differential equations. However if, for example, we consider a biochemical pathway the constituent chemical species and hence the pathway structure are seldom fully characterised. In addition it is often impossible to obtain values of the rates of activation or decay which form the free parameters of the mathematical model. The system model in many cases is therefore not fully characterised either in terms of structure or the values which parameters take. This uncertainty must be accounted for in a systematic manner when the model is used in simulation or predictive mode to safeguard against reaching conclusions about system characteristics that are unwarranted, or in making predictions that are unjustifiably optimistic given the uncertainty about the model. The Bayesian inferential methodology provides a coherent framework with which to characterise and propagate uncertainty in such mechanistic models and this paper provides an introduction to Bayesian methodology as applied to system models represented as differential equations. [Copyright &y& Elsevier]

Abstract: Fraud detection in the telecommunications industry requires the identification of a small fraction of fraudulent calls from the high volume of call traffic. This presents a significant research challenge in the design of efficient and effective algorithms to combat telecommunications fraud. This paper employs Latent Dirichlet Allocation (LDA) to build user profile signatures and assumes that any significant unexplainable deviations from the normal activity of an individual user is strongly correlated with fraudulent activity. The user activity is represented as a probability distribution over call features which surmises the user’s calling behaviour. This probability distribution is derived from LDA which can accurately describe user profiles by combining different classes of distributions. To score calls we compare the likelihood of the user generating a call versus a fraudster generating the same call. Our experiments demonstrate that using such a probability distribution and employing even a rough profile of a fraudster’s activity ameliorates the detection of fraudulent calls. Our method is computationally efficient and can scale up to a realistic real time detection. [Copyright &y& Elsevier]

It is well known in the statistics literature that augmenting binary and polychotomous response models with gaussian latent variables enables exact Bayesian analysis via Gibbs sampling from the parameter posterior. By adopting such a data augmentation strategy, dispensing with priors over regression coefficients in favor of gaussian process (GP) priors over functions, and employing variational approximations to the full posterior, we obtain efficient computational methods for GP classification in the multiclass setting.1 The model augmentation with additional latent variables ensures full a posteriori class coupling while retaining the simple a priori independent GP covariance structure from which sparse approximations, such as multiclass informative vector machines (IVM), emerge in a natural and straightforward manner. This is the first time that a fully variational Bayesian treatment for multiclass GP classification has been developed without having to resort to additional explicit approximations to the nongaussian likelihood term. Empirical comparisons with exact analysis use Markov Chain Monte Carlo (MCMC) and Laplace approximations illustrate the utility of the variational approximation as a computationally economic alternative to full MCMC and it is shown to be more accurate than the Laplace approximation. [ABSTRACT FROM AUTHOR]

Spectral clustering methods were proposed recently which rely on the eigenvalue decomposition of an affinity matrix. In this letter, the affinity matrix is created from the elements of a nonparametric density estimator and then decomposed to obtain posterior probabilities of class membership. Hyperparameters are selected using standard cross-validation methods. [ABSTRACT FROM AUTHOR]

This paper considers the application of a recently proposed L2 optimal non-parametric reduced set density estimator to novelty detection and binary classification and provides empirical comparisons with other forms of density estimation as well as support vector machines. [Copyright &y& Elsevier]

Automated currency validation requires a decision to be made regarding the authenticity of a banknote presented to the validation system. This decision often has to be made with little or no information regarding the characteristics of possible counterfeits as is the case for issues of new currency. A method for automated currency validation is presented which segments the whole banknote into different regions, builds individual classifiers on each region and then combines a small subset of the region specific classifiers to provide an overall decision. The segmentation and combination of region specific classifiers to provide optimized false positive and false negative rates is achieved by employing a genetic algorithm. Experiments based on high value notes of Sterling currency were carried out to assess the effectiveness of the proposed solution. [Copyright &y& Elsevier]

Presents the Reduced Set Density Estimator that provides a kernel-based density estimator which employs a small percentage of the available data sample and is optimal in the L[sub 2] sense. Method that provides similar levels of performance accuracy and sparseness of representation as Support Vector Machine density estimation.

This paper is concerned with learning dense low-dimensional representations of high-dimensional positive data. The positive data may be continuous, discrete binary or count based. In addition to the low-dimensional data model, a topographic ordering of the representation is desired. The primary motivation for this work is the requirement for a low-dimensional interpretation of sparse vector space models of text documents which may take the form of binary, count based or real multivariate data. The generative topographic mapping (GTM) was developed and introduced as a principled alternative to the self-organising map for, principally, visualising high-dimensional continuous data. The GTM is one method by which a topographically organised low-dimensional data representation may be realised. There are many cases where the observation data is discrete and the application of methods developed specifically for continuous data is inappropriate. Based on the continuous GTM data model a non-linear latent variable model for modelling high-dimensional binary data is presented. The non-negative factorisation of a positive matrix which ensures a topographic ordering of the constituent factors is also presented as a principled yet non-probabilistic alternative to the GTM model. Experimental demonstrations of both methods are provided based on representing binary coded handwritten digits and the topographic organisation and visualisation of a collection of text based documents. [Copyright &y& Elsevier]

Presents a study that developed a method for both the unsupervised partitioning of a sample of data and the estimation of the possible number of inherent clusters which generate the data. Background on sum-of-squares clustering in data space; Details on the clustering of data in a nonlinear feature space; Stochastic optimization in estimating the number of inherent clusters within a data sample; Estimation of the number of clusters using the kernel matrix.

Kernel principal component analysis has been introduced as a method of extracting a set of orthonormal nonlinear features from multivariate data, and many impressive applications are being reported within the literature. This article presents the view that the eigenvalue decomposition of a kernel matrix can also provide the discrete expansion coefficients required for a nonparametric orthogonal series density estimator. In addition to providing novel insights into nonparametric density estimation, this article provides an intuitively appealing interpretation for the nonlinear features extracted from data using kernel principal component analysis. [ABSTRACT FROM AUTHOR]

Presents a study which developed a nonlinear latent variable model for the topographic organization and subsequent visualization of multivariate binary data. Theoretical background; Methods used; Findings and discussion.

An expectation-maximization algorithm for learning sparse and overcomplete data representations is presented. The proposed algorithm exploits a variational approximation to a range of heavy-tailed distributions whose limit is the Laplacian. A rigorous lower bound on the sparse prior distribution is derived, which enables the analytic marginalization of a lower bound on the data likelihood. This lower bound enables the development of an expectation-maximization algorithm for learning the overcomplete basis vectors and inferring the most probable basis coefficients. [ABSTRACT FROM AUTHOR]

Discusses a study which presented a Generative Topographic Mapping, a general tool for data analysis and visualization by means of topographic organization and clustering. General formulation of the probabilistic generative model; Description of the Expectation Maximization algorithm; Implication of data sparseness in generative class modeling; Particular cases in modeling discrete data.

In this paper, we propose the application of the Kernel Principal Component Analysis (PCA) technique for feature selection in a high-dimensional feature space, where input variables are mapped by a Gaussian kernel. The extracted features are employed in the regression problems of chaotic Mackey–Glass time-series prediction in a noisy environment and estimating human signal detection performance from brain event-related potentials elicited by task relevant signals. We compared results obtained using either Kernel PCA or linear PCA as data preprocessing steps. On the human signal detection task, we report the superiority of Kernel PCA feature extraction over linear PCA. Similar to linear PCA, we demonstrate de-noising of the original data by the appropriate selection of various nonlinear principal components. The theoretical relation and experimental comparison of Kernel Principal Components Regression, Kernel Ridge Regression and ε-insensitive Support Vector Regression is also provided. [ABSTRACT FROM AUTHOR]

The proposal of considering nonlinear principal component analysis as a kernel eigenvalue problem has provided an extremely powerful method of extracting nonlinear features for a number of classification and regression applications. Whereas the utilization of Mercer kernels makes the problem of computing principal components in, possibly, infinite-dimensional feature spaces tractable, there are still the attendant numerical problems of diagonalizing large matrices. In this contribution, we propose an expectation-maximization approach for performing kernel principal component analysis and show this to be a computationally efficient method, especially when the number of data points is large. [ABSTRACT FROM AUTHOR]

Presents a study of unsupervised exploratory data analysis and independent component analysis by a common neural network model. Presentations of an unsupervised learning algorithm; Capabilities of the algorithm in the study; What is confirmed by the experimental results of the study.

An extension of the infomax algorithm of Bell and Sejnowski (1995) is presented that is able blindly to separate mixed signals with sub- and supergaussian source distributions. This was achieved by using a simple type of learning rule first derived by Girolami (1997) by choosing negentropy as a projection pursuit index. Parameterized probability distributions that have sub- and supergaussian regimes were used to derive a general learning rule that preserves the simple architecture proposed by Bell and Sejnowski (1995), is optimized using the natural gradient by Amari (1998), and uses the stability analysis of Cardoso and Laheld (1996) to switch between sub- and supergaussian regimes. We demonstrate that the extended infomax algorithm is able to separate 20 sources with a variety of source distributions easily. Applied to high-dimensional data from electroencephalographic recordings, it is effective at separating artifacts such as eye blinks and line noise from weaker electrical signals that arise from sources in the brain. [ABSTRACT FROM AUTHOR]

This article develops an extended independent component analysis algorithm for mixtures of arbitrary subgaussian and supergaussian sources. The gaussian mixture model of Pearson is employed in deriving a closedform generic score function for strictly subgaussian sources. This is combined with the score function for a unimodal supergaussian density to provide a computationally simple yet powerful algorithm for performing independent component analysis on arbitrary mixtures of nongaussian sources. [ABSTRACT FROM AUTHOR]

The increased availability of observation data from engineering systems in operation poses the question of how to incorporate this data into finite element models. To this end, we propose a novel statistical construction of the finite element method that provides the means of synthesising measurement data and finite element models. The Bayesian statistical framework is adopted to treat all the uncertainties present in the data, the mathematical model and its finite element discretisation. From the outset, we postulate a statistical generating model which additively decomposes data into a finite element, a model misspecification and a noise component. Each of the components may be uncertain and is considered as a random variable with a respective prior probability density. The prior of the finite element component is given by a conventional stochastic forward problem. The prior probabilities of the model misspecification and measurement noise, without loss of generality, are assumed to have a zero-mean and a known covariance structure. Our proposed statistical model is hierarchical in the sense that each of the three random components may depend on one or more non-observable random hyperparameters with their own corresponding probability densities. We use Bayes rule to infer the posterior densities of the three random components and the hyperparameters from their known prior densities and a data dependent likelihood function. Because of the hierarchical structure of our statistical model, Bayes rule is applied on three different levels in turn. On level one, we determine the posterior densities of the finite element component and the true system response using the prior finite element density given by the forward problem and the data likelihood. In this step, approximating the prior finite element density with a multivariate Gaussian distribution allows us to obtain a closed-form expression for the posterior. On the next level, we infer the hyperparameter posterior densities from their respective priors and the marginal likelihood of the first inference problem. These posteriors are sampled numerically using the Markov chain Monte Carlo (MCMC) method. Finally, on level three we use Bayes rule to choose the most suitable finite element model in light of the observed data by computing the respective model posteriors. We demonstrate the application and versatility of statFEM with one and two-dimensional examples. • Statistical FE approach for synthesis of observation data and FE models is proposed. • A data-generating process combining FE model, model mismatch and data is assumed. • All random variables and hyperparameters are learned using Bayesian inference. • Closed-form expressions for the posterior FE and true system probabilities are derived. • Convergence of the posterior probabilities with increasing data is demonstrated. [ABSTRACT FROM AUTHOR]

• Clustering log data is fundamental for providing insights into many critical areas. • Feedback such as labels and pairwise constraints can improve log clustering quality. • A principled way to summarize the data can be used to deal with a high volume of data. Machine-generated log data can provide valuable insights into many critical areas such as system failures, network security, and performance optimization. The increasing prominence of this data in both volume and complexity requires data mining approaches that are both scalable and flexible. In this paper, we propose a new approach for clustering machine-generated logs which contains a novel combination of the use of the coreset with user feedback. The coreset allows us to efficiently summarize the data in a principled manner such that performance after fitting model parameters on the coreset is similar to the performance that would have been achieved with the full dataset. Furthermore, the formal approach we propose allows users to incorporate two different types of feedback, in the forms of labels and pairwise constraints, to further improve results and better deal with the increasing complexity and variety of log datasets. [ABSTRACT FROM AUTHOR]

Gaussian processes are ubiquitous in machine learning, statistics, and applied mathematics. They provide a flexible modelling framework for approximating functions, whilst simultaneously quantifying uncertainty. However, this is only true when the model is well-specified, which is often not the case in practice. In this paper, we study the properties of Gaussian process means when the smoothness of the model and the likelihood function are misspecified. In this setting, an important theoretical question of practical relevance is how accurate the Gaussian process approximations will be given the chosen model and the extent of the misspecification. The answer to this problem is particularly useful since it can inform our choice of model and experimental design. In particular, we describe how the experimental design and choice of kernel and kernel hyperparameters can be adapted to alleviate model misspecification. [ABSTRACT FROM AUTHOR]

To mitigate the current COVID-19 pandemic, policy makers at the Greater London Authority, the regional governance body of London, UK, are reliant upon prompt, accurate and actionable estimations of lockdown and social distancing policy adherence. Transport for London, the local transportation department, reports they implemented over 700 interventions such as greater signage and expansion of pedestrian zoning at the height of the pandemic's first wave with our platform providing key data for those decisions. Large well-defined heterogeneous compositions of pedestrian footfall and physical proximity are difficult to acquire, yet necessary to monitor city-wide activity (busyness) and consequently discern actionable policy decisions. To meet this challenge, we leverage our existing large-scale data processing urban air quality machine learning infrastructure to process over 900 camera feeds in near real-time to generate new estimates of social distancing adherence, group detection and camera stability. In this work, we describe our development and deployment of a computer vision and machine learning pipeline. It provides near immediate sampling and contextualization of activity and physical distancing on the streets of London via live traffic camera feeds. We introduce a platform for inspecting, calibrating and improving upon existing methods, describe the active deployment on real-time feeds and provide analysis over an 18 month period. [ABSTRACT FROM AUTHOR]

At the heart of structural engineering research is the use of data obtained from physical structures such as bridges, viaducts and buildings. These data can represent how the structure responds to various stimuli over time when in operation. Many models have been proposed in literature to represent such data, such as linear statistical models. Based upon these models, the health of the structure is reasoned about, e.g. through damage indices, changes in likelihood and statistical parameter estimates. On the other hand, physics-based models are typically used when designing structures to predict how the structure will respond to operational stimuli. These models represent how the structure responds to stimuli under idealised conditions. What remains unclear in the literature is how to combine the observed data with information from the idealised physics-based model into a model that describes the responses of the operational structure. This paper introduces a new approach which fuses together observed data from a physical structure during operation and information from a mathematical model. The observed data are combined with data simulated from the physics-based model using a multi-output Gaussian process formulation. The novelty of this method is how the information from observed data and the physics-based model is balanced to obtain a representative model of the structures response to stimuli. We present our method using data obtained from a fibre-optic sensor network installed on experimental railway sleepers. The curvature of the sleeper at sensor and also non-sensor locations is modelled, guided by the mathematical representation. We discuss how this approach can be used to reason about changes in the structures behaviour over time using simulations and experimental data. The results show that the methodology can accurately detect such changes. They also indicate that the methodology can infer information about changes in the parameters within the physics-based model, including those governing components of the structure not measured directly by sensors such as the ballast foundation. • Probabilistic models can be used to balance information obtained from physics-based models and observed data. • Balancing is implemented by maximizing the predictive performance of observed test data-points. • Physics-based models can improve the inference of the structural response in areas of the domain which are unmeasured. • The proposed information synthesis can infer model parameters that govern structural components that cannot be measured. • Experimental application of the proposed methodology, detecting underlying changes in model parameters resulting from damage. [ABSTRACT FROM AUTHOR]

Motivation: There are several levels of uncertainty involved in the mathematical modelling of biochemical systems. There often may be a degree of uncertainty about the values of kinetic parameters, about the general structure of the model and about the behaviour of biochemical species which cannot be observed directly. The methods of Bayesian inference provide a consistent framework for modelling and predicting in these uncertain conditions. We present a software package for applying the Bayesian inferential methodology to problems in systems biology. [ABSTRACT FROM PUBLISHER]

We propose a determinant-free approach for simulation-based Bayesian inference in high-dimensional Gaussian models. We introduce auxiliary variables with covariance equal to the inverse covariance of the model. The joint probability of the auxiliary model can be computed without evaluating determinants, which are often hard to compute in high dimensions. We develop a Markov chain Monte Carlo sampling scheme for the auxiliary model that requires no more than the application of inverse-matrix-square-roots and the solution of linear systems. These operations can be performed at large scales with rational approximations. We provide an empirical study on both synthetic and real-world data for sparse Gaussian processes and for large-scale Gaussian Markov random fields. Copyright © 2017 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

A Bayesian computational approach is presented to provide a multi-resolution estimate of an unknown spatially varying parameter from indirect measurement data. In particular, we are interested in spatially varying parameters with multiscale characteristics. In our work, we consider the challenge of not knowing the characteristic length scale(s) of the unknown a priori, and present an algorithm for on-the-fly scale determination. Our approach is based on representing the spatial field with a wavelet expansion. Wavelet basis functions are hierarchically structured, localized in both spatial and frequency domains and tend to provide sparse representations in that a large number of wavelet coefficients are approximately zero. For these reasons, wavelet bases are suitable for representing permeability fields with non-trivial correlation structures. Moreover, the intra-scale correlations between wavelet coefficients form a quadtree, and this structure is exploited to identify additional basis functions to refine the model. Bayesian inference is performed using a sequential Monte Carlo (SMC) sampler with a Markov Chain Monte Carlo (MCMC) transition kernel. The SMC sampler is used to move between posterior densities defined on different scales, thereby providing a computationally efficient method for adaptive refinement of the wavelet representation. We gain insight from the marginal likelihoods, by computing Bayes factors, for model comparison and model selection. The marginal likelihoods provide a termination criterion for our scale determination algorithm. The Bayesian computational approach is rather general and applicable to several inverse problems concerning the estimation of a spatially varying parameter. The approach is demonstrated with permeability estimation for groundwater flow using pressure sensor measurements. [ABSTRACT FROM AUTHOR]

Approximation of the model evidence is well known to be challenging. One promising approach is based on thermodynamic integration, but a key concern is that the thermodynamic integral can suffer from high variability in many applications. This article considers the reduction of variance that can be achieved by exploiting control variates in this setting. Our methodology applies whenever the gradient of both the log-likelihood and the log-prior with respect to the parameters can be efficiently evaluated. Results obtained on regression models and popular benchmark datasets demonstrate a significant and sometimes dramatic reduction in estimator variance and provide insight into the wider applicability of control variates to evidence estimation. Supplementary materials for this article are available online. [ABSTRACT FROM PUBLISHER]

Incorporating unstructured data into physical models is a challenging problem that is emerging in data assimilation. Traditional approaches focus on well-defined observation operators whose functional forms are typically assumed to be known. This prevents these methods from achieving a consistent model-data synthesis in configurations where the mapping from data-space to model-space is unknown. To address these shortcomings, in this paper we develop a physics-informed dynamical variational autoencoder (Φ-DVAE) to embed diverse data streams into time-evolving physical systems described by differential equations. Our approach combines a standard, possibly nonlinear, filter for the latent state-space model and a VAE, to assimilate the unstructured data into the latent dynamical system. Unstructured data, in our example systems, comes in the form of video data and velocity field measurements, however the methodology is suitably generic to allow for arbitrary unknown observation operators. A variational Bayesian framework is used for the joint estimation of the encoding, latent states, and unknown system parameters. To demonstrate the method, we provide case studies with the Lorenz-63 ordinary differential equation, and the advection and Korteweg-de Vries partial differential equations. Our results, with synthetic data, show that Φ-DVAE provides a data efficient dynamics encoding methodology which is competitive with standard approaches. Unknown parameters are recovered with uncertainty quantification, and unseen data are accurately predicted. • Bayesian inference methodology for unstructured data assimilation. • Variational autoencoder embeds data to observations of latent differential equation. • Statistical FEM construction and parameter estimation account for misspecification. • Demonstrated on Lorenz-63, advection and Korteweg-de Vries differential equations. • Embedding physical prior knowledge produces data-efficient learning. [ABSTRACT FROM AUTHOR]

• A novel generative broad Bayesian imputer for missing data imputation is proposed. • An augmentable broad Bayesian learning network is developed. • The learning network can be augmented progressively to adopt reconfiguration. • The proposed imputer provides a universal tool for missing data imputation. • Optimal imputation with quantified uncertainty can be obtained. Generative broad Bayesian (GBB) imputer, a novel nonparametric data-driven tool for missing data imputation with uncertainty quantification, is proposed. The proposed imputer aims to generate missing data in an iterative manner based on an augmentable broad Bayesian learning network. The procedure consists of the preparatory and tuning phase. The preparatory phase provides preliminary imputation of the missing data to develop a complete dataset. The tuning phase refines the accuracy of the imputation results based on the augmented learning network. There are three appealing features of the proposed GBB imputer: (i) the nonparametric generative scheme provides a universal tool for missing data imputation without constraints on the type of data attribution, missing data pattern, or requirement of the prior information about the dataset; (ii) the quantified uncertainty of the imputation results reflects the associated reliability and provides a rational termination indicator for the iterative imputation procedure; and (iii) the learning network can be augmented progressively to adopt architectural reconfigurations based on the inherited information of the trained network for efficient imputation. To demonstrate the efficacy and applicability of the proposed GBB imputer, we present two simulated examples under various scenarios and a case study with the achieved in-situ seismic records of the 2016 M w 6.5 Norcia earthquake. [ABSTRACT FROM AUTHOR]

We deliver a call to arms for probabilistic numerical methods: algorithms for numerical tasks, including linear algebra, integration, optimization and solving differential equations, that return uncertainties in their calculations. Such uncertainties, arising from the loss of precision induced by numerical calculation with limited time or hardware, are important for much contemporary science and industry. Within applications such as climate science and astrophysics, the need to make decisions on the basis of computations with large and complex data have led to a renewed focus on the management of numerical uncertainty. We describe how several seminal classic numerical methods can be interpreted naturally as probabilistic inference. We then show that the probabilistic view suggests new algorithms that can flexibly be adapted to suit application specifics, while delivering improved empirical performance. We provide concrete illustrations of the benefits of probabilistic numeric algorithms on real scientific problems from astrometry and astronomical imaging, while highlighting open problems with these new algorithms. Finally, we describe how probabilistic numerical methods provide a coherent framework for identifying the uncertainty in calculations performed with a combination of numerical algorithms (e.g. both numerical optimizers and differential equation solvers), potentially allowing the diagnosis (and control) of error sources in computations. [ABSTRACT FROM AUTHOR]

Significant funds have been allocated to maintain road networks each year in developed countries. Performance prediction is crucial for pavement management systems to adjust working plans and budget allocation. As a Bayesian nonparametric method, Gaussian process regression (GPR) is powerful in predicting nonlinear time series and quantifying uncertainty. However, it remains computationally intensive and fails to adapt to the time-varying characteristics. To address such issues, a dynamic GPR model is proposed for probabilistic prediction of the International Roughness Index (IRI) for flexible pavements. A moving window strategy is developed to substantially shrink the size of training data, which effectively alleviates computational cost and thus leads to a dynamic GPR. A genetic algorithm is then adopted to determine the optimal window size by considering the trade-off between computational efficiency and accuracy. A dataset acquired from Long-Term Pavement Performance (LTPP) is used to demonstrate the feasibility of the dynamic GPR. Its performance is compared to traditional GPR as well as dynamic and static Bayesian linear regression (BLR) models. The comparison results indicate that the proposed dynamic GPR can increase the accuracy by 0.86, 1.52, and 2.27 times for dynamic BLR, static GPR, and static BLR, respectively. It exhibits the best results in terms of accuracy and uncertainty metrics due to its nonlinear modelling and time-varying ability. • A dynamic Gaussian process regression (GPR) model is proposed for probabilistic prediction of pavement conditions. • A moving window strategy is developed to effectively alleviate the computational cost. • A genetic algorithm is adopted to determine the optimal window size. • The traditional GPR as well as dynamic and static Bayesian linear regression models are employed for comparison. [ABSTRACT FROM AUTHOR]

Background: Detection of sentences that describe protein-protein interactions (PPIs) in biomedical publications is a challenging and unresolved pattern recognition problem. Many state-of-the-art approaches for this task employ kernel classification methods, in particular support vector machines (SVMs). In this work we propose a novel data integration approach that utilises semantic kernels and a kernel classification method that is a probabilistic analogue to SVMs. Semantic kernels are created from statistical information gathered from large amounts of unlabelled text using lexical semantic models. Several semantic kernels are then fused into an overall composite classification space. In this initial study, we use simple features in order to examine whether the use of combinations of kernels constructed using word-based semantic models can improve PPI sentence detection. Results: We show that combinations of semantic kernels lead to statistically significant improvements in recognition rates and receiver operating characteristic (ROC) scores over the plain Gaussian kernel, when applied to a well-known labelled collection of abstracts. The proposed kernel composition method also allows us to automatically infer the most discriminative kernels. Conclusions: The results from this paper indicate that using semantic information from unlabelled text, and combinations of such information, can be valuable for classification of short texts such as PPI sentences. This study, however, is only a first step in evaluation of semantic kernels and probabilistic multiple kernel learning in the context of PPI detection. The method described herein is modular, and can be applied with a variety of feature types, kernels, and semantic models, in order to facilitate full extraction of interacting proteins. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the sparsity and recognition capabilities of two approximate Bayesian classification algorithms, the multiclass multi-kernel relevance vector machines (mRVMs) that have been recently proposed. We provide an insight into the behavior of the mRVM models by performing a wide experimentation on a large range of real-world datasets. Furthermore, we monitor various model fitting characteristics that identify the predictive nature of the proposed methods and compare against existing classification techniques. By introducing novel convergence measures, sample selection strategies and model improvements, it is demonstrated that mRVMs can produce state-of-the-art results on multiclass discrimination problems. In addition, this is achieved by utilizing only a very small fraction of the available observation data. [ABSTRACT FROM PUBLISHER]

• A novel nonparametric reliability-based output feedback control algorithm is proposed for semi-active structural control. • An adaptive broad learning network is developed to formulate the control strategy using the clipped-optimal control technique. • It is developed adaptively to adopt additional optimal training input–output data pairs. • It incorporates both predicted failure probability and the uncertainty of the underlying structure. • The proposed algorithm provides optimal and robust control regardless of the uncertainty of the structural parameters, external excitation, and structural damage conditions. We propose a novel algorithm for dynamic response suppression via semi-active control devices, which we refer to as broad learning, robust, semi-active control (BLRSAC). To configure the semi-active controller, a nonparametric reliability-based output feedback control strategy is introduced. In particular, an adaptive broad learning network is developed to formulate the control strategy using the clipped-optimal control technique. The learning network is augmented incrementally to adopt additional training data based on the inherited information of the trained learning network. By utilizing a robust failure probability, the training dataset is obtained adaptively to include the training input–output pairs with optimal structural control performance. The robust failure probability we propose incorporates both predicted failure probability and the uncertainty of the underlying structure. Therefore, the resultant control strategy can handle the inevitable uncertainty of the actual control situation to achieve optimal structural control. To examine the efficacy of the proposed BLRSAC algorithm, illustrative examples of a shear building and a three-dimensional braced frame under various external excitation and structural damaging conditions are presented. [ABSTRACT FROM AUTHOR]

Abstract: Classifying electroencephalography (EEG) signals is an important step for proceeding EEG-based brain computer interfaces (BCI). Currently, kernel based methods such as the support vector machine (SVM) are considered the state-of-the-art methods for this problem. In this paper, we apply Gaussian process (GP) classification to binary discrimination of motor imagery EEG data. Compared with the SVM, GP based methods naturally provide probability outputs for identifying a trusted prediction which can be used for post-processing in a BCI. Experimental results show that the classification methods based on a GP perform similarly to kernel logistic regression and probabilistic SVM in terms of predictive likelihood, but outperform SVM and K-nearest neighbor (KNN) in terms of 0–1 loss class prediction error. [Copyright &y& Elsevier]