Search

We introduce neural information field filter, a Bayesian state and parameter estimation method for high-dimensional nonlinear dynamical systems given large measurement datasets. Solving such a problem using traditional methods, such as Kalman and particle filters, is computationally expensive. Information field theory is a Bayesian approach that can efficiently reconstruct dynamical model state paths and calibrate model parameters from noisy measurement data. To apply the method, we parameterize the time evolution state path using the span of a finite linear basis. The existing method has to reparameterize the state path by initial states to satisfy the initial condition. Designing an expressive yet simple linear basis before knowing the true state path is crucial for inference accuracy but challenging. Moreover, reparameterizing the state path using the initial state is easy to perform for a linear basis, but is nontrivial for more complex and expressive function parameterizations, such as neural networks. The objective of this paper is to simplify and enrich the class of state path parameterizations using neural networks for the information field theory approach. To this end, we propose a generalized physics-informed conditional prior using an auxiliary initial state. We show the existing reparameterization is a special case. We parameterize the state path using a residual neural network that consists of a linear basis function and a Fourier encoding fully connected neural network residual function. The residual function aims to correct the error of the linear basis function. To sample from the intractable posterior distribution, we develop an optimization algorithm, nested stochastic variational inference, and a sampling algorithm, nested preconditioned stochastic gradient Langevin dynamics. A series of numerical and experimental examples verify and validate the proposed method.

Non-invasive flow measurement techniques, such as particle tracking velocimetry, resolve 3D velocity fields by pairing tracer particle positions in successive time steps. These trajectories are crucial for evaluating physical quantities like vorticity, shear stress, pressure, and coherent structures. Traditional approaches deterministically reconstruct particle positions and extract particle tracks using tracking algorithms. However, reliable track estimation is challenging due to measurement noise caused by high particle density, particle image overlap, and falsely reconstructed 3D particle positions. To overcome this challenge, probabilistic approaches quantify the epistemic uncertainty in particle positions, typically using a Gaussian probability distribution. However, the standard deterministic tracking algorithms relying on nearest-neighbor search do not directly extend to the probabilistic setting. Moreover, such algorithms do not necessarily find globally consistent solutions robust to reconstruction errors. This paper aims to develop a globally consistent nearest-neighborhood algorithm that robustly extracts stochastic particle tracks from the reconstructed Gaussian particle distributions in all frames. Our tracking algorithm relies on the unbalanced optimal transport theory in the metric space of Gaussian measures. Specifically, we optimize a binary transport plan for efficiently moving the Gaussian distributions of reconstructed particle positions between time frames. We achieve this by computing the partial Wasserstein distance in the metric space of Gaussian measures. Our tracking algorithm is robust to position reconstruction errors since it automatically detects the number of particles that should be matched through hyperparameter optimization. Finally, we validate our method using an in vitro flow experiment using a 3D-printed cerebral aneurysm.

We derive conditions in which inverse problems posed under an information field theory (IFT) framework have unique solutions. While the theorems here apply to IFT inverse problems in general, we place a special focus on the problem of identifying model-form error. Due to the continued increase in popularity of physics-informed methods, tools which can validate the form of the physics chosen under a physics-informed framework are desirable. Using IFT, we pose the problem of identifying model-form error as a Bayesian inverse problem. The IFT framework lets us define a physics-informed prior over fields, where a parameter, which we call the trust, measures our belief in the physical model by scaling the spread of this prior. In principle, smaller values of trust cause the prior to appear flat, representing a larger degree of uncertainty about the physics. To detect model-form error, we infer the trust as part of an inverse problem and study the limiting behavior. Using the Gaussian random field interpretation of IFT, we show how identifying the trust becomes a well-posed inverse problem. We provide an example where the physics are assumed to be the Poisson equation and study the effect of model-form error on the trust. We find that a correct model leads to infinite trust, and under model-form error, physics which are closer to the ground truth lead to larger values of the trust., Comment: 17 pages. Will be presented at SIAM Conference on Uncertainty Quantification 2024

The area-normalized change of mass ($\Delta$m/A) with time during the oxidation of metallic alloys is commonly used to assess oxidation resistance. Analyses of such data can also aid in evaluating underlying oxidation mechanisms. We performed an exhaustive literature search and digitized normalized mass change vs. time data for 407 alloys. To maximize the impact of these and future mass uptake data, we developed and published an open, online, computational workflow that fits the data to various models of oxidation kinetics, uses Bayesian statistics for model selection, and makes the raw data and model parameters available via a queryable database. The tool, Refractory Oxidation Database (https://nanohub.org/tools/refoxdb/), uses nanoHUB's Sim2Ls to make the workflow and data (including metadata) findable, accessible, interoperable, and reusable (FAIR). We find that the models selected by the original authors do not match the most likely one according to the Bayesian information criterion (BIC) in 71% of the cases. Further, in 56% of the cases, the published model was not even in the top 3 models according to the BIC. These numbers were obtained assuming an experimental noise of 2.5% of the mass gain range, a smaller noise leads to more discrepancies. The RefOxDB tool is open access and researchers can add their own raw data (those to be included in future publications, as well as negative results) for analysis and to share their work with the community. Such consistent and systematic analysis of open, community generated data can significantly accelerate the development of machine-learning models for oxidation behavior and assist in the understanding and improvement of oxidation resistance.

Many natural materials exhibit highly complex, nonlinear, anisotropic, and heterogeneous mechanical properties. Recently, it has been demonstrated that data-driven strain energy functions possess the flexibility to capture the behavior of these complex materials with high accuracy while satisfying physics-based constraints. However, most of these approaches disregard the uncertainty in the estimates and the spatial heterogeneity of these materials. In this work, we leverage recent advances in generative models to address these issues. We use as building block neural ordinary equations (NODE) that -- by construction -- create polyconvex strain energy functions, a key property of realistic hyperelastic material models. We combine this approach with probabilistic diffusion models to generate new samples of strain energy functions. This technique allows us to sample a vector of Gaussian white noise and translate it to NODE parameters thereby representing plausible strain energy functions. We extend our approach to spatially correlated diffusion resulting in heterogeneous material properties for arbitrary geometries. We extensively test our method with synthetic and experimental data on biological tissues and run finite element simulations with various degrees of spatial heterogeneity. We believe this approach is a major step forward including uncertainty in predictive, data-driven models of hyperelasticity, Comment: 22 pages, 11 figures

Dynamical system state estimation and parameter calibration problems are ubiquitous across science and engineering. Bayesian approaches to the problem are the gold standard as they allow for the quantification of uncertainties and enable the seamless fusion of different experimental modalities. When the dynamics are discrete and stochastic, one may employ powerful techniques such as Kalman, particle, or variational filters. Practitioners commonly apply these methods to continuous-time, deterministic dynamical systems after discretizing the dynamics and introducing fictitious transition probabilities. However, approaches based on time-discretization suffer from the curse of dimensionality since the number of random variables grows linearly with the number of time-steps. Furthermore, the introduction of fictitious transition probabilities is an unsatisfactory solution because it increases the number of model parameters and may lead to inference bias. To address these drawbacks, the objective of this paper is to develop a scalable Bayesian approach to state and parameter estimation suitable for continuous-time, deterministic dynamical systems. Our methodology builds upon information field theory. Specifically, we construct a physics-informed prior probability measure on the function space of system responses so that functions that satisfy the physics are more likely. This prior allows us to quantify model form errors. We connect the system's response to observations through a probabilistic model of the measurement process. The joint posterior over the system responses and all parameters is given by Bayes' rule. To approximate the intractable posterior, we develop a stochastic variational inference algorithm. In summary, the developed methodology offers a powerful framework for Bayesian estimation in dynamical systems.

Inverse problems, i.e., estimating parameters of physical models from experimental data, are ubiquitous in science and engineering. The Bayesian formulation is the gold standard because it alleviates ill-posedness issues and quantifies epistemic uncertainty. Since analytical posteriors are not typically available, one resorts to Markov chain Monte Carlo sampling or approximate variational inference. However, inference needs to be rerun from scratch for each new set of data. This drawback limits the applicability of the Bayesian formulation to real-time settings, e.g., health monitoring of engineered systems, and medical diagnosis. The objective of this paper is to develop a methodology that enables real-time inference by learning the Bayesian inverse map, i.e., the map from data to posteriors. Our approach is as follows. We parameterize the posterior distribution as a function of data. This work outlines two distinct approaches to do this. The first method involves parameterizing the posterior using an amortized full-rank Gaussian guide, implemented through neural networks. The second method utilizes a Conditional Normalizing Flow guide, employing conditional invertible neural networks for cases where the target posterior is arbitrarily complex. In both approaches, we learn the network parameters by amortized variational inference which involves maximizing the expectation of evidence lower bound over all possible datasets compatible with the model. We demonstrate our approach by solving a set of benchmark problems from science and engineering. Our results show that the posterior estimates of our approach are in agreement with the corresponding ground truth obtained by Markov chain Monte Carlo. Once trained, our approach provides the posterior distribution for a given observation just at the cost of a forward pass of the neural network.

Data-driven approaches coupled with physical knowledge are powerful techniques to model systems. The goal of such models is to efficiently solve for the underlying field by combining measurements with known physical laws. As many systems contain unknown elements, such as missing parameters, noisy data, or incomplete physical laws, this is widely approached as an uncertainty quantification problem. The common techniques to handle all the variables typically depend on the numerical scheme used to approximate the posterior, and it is desirable to have a method which is independent of any such discretization. Information field theory (IFT) provides the tools necessary to perform statistics over fields that are not necessarily Gaussian. We extend IFT to physics-informed IFT (PIFT) by encoding the functional priors with information about the physical laws which describe the field. The posteriors derived from this PIFT remain independent of any numerical scheme and can capture multiple modes, allowing for the solution of problems which are ill-posed. We demonstrate our approach through an analytical example involving the Klein-Gordon equation. We then develop a variant of stochastic gradient Langevin dynamics to draw samples from the joint posterior over the field and model parameters. We apply our method to numerical examples with various degrees of model-form error and to inverse problems involving nonlinear differential equations. As an addendum, the method is equipped with a metric which allows the posterior to automatically quantify model-form uncertainty. Because of this, our numerical experiments show that the method remains robust to even an incorrect representation of the physics given sufficient data. We numerically demonstrate that the method correctly identifies when the physics cannot be trusted, in which case it automatically treats learning the field as a regression problem., Comment: 32 pages, 8 figures. Published in Journal of Computational Physics

A turbocharger plays an essential part in reducing emissions and increasing the fuel efficiency of road vehicles. The pulsating flow of exhaust gases, along with high heat exchange from the turbocharger casing, makes developing control-oriented models difficult. Several researchers have used maps provided by manufacturers to solve this problem. These maps often fail to incorporate any heat transfer effects and are unsuitable for wide operating regions. Also, with the availability of more and better sensor data, there is a need for a method that can exploit this to obtain a better predictive model. Koopman approaches rely on the observation that one can lift the nonlinear dynamics of the turbine into an infinite-dimensional function space over which dynamics are linear. The objective of this paper is to develop a model to predict the transient and steady-state behavior of the turbine using the Koopman operator which can be helpful for control design and analysis. Our approach is as follows. We use experimental data from a Cummins heavy-duty diesel engine to develop a turbine model using Extended Dynamic Mode Decomposition, which approximates the action of the Koopman operator on a finite-dimensional subspace of the space of observables. The results demonstrate superior performance compared to a tuned nonlinear autoregressive network with an exogenous input model widely used in the literature. The performance of these two models is analyzed based on their ability to predict turbine transient and steady-state behavior., Comment: Accepted at Modeling, Estimation, and Control Conference 2022

We present a Bayesian methodology to infer the elastic modulus of the constituent polymer and the fiber orientation state in a short-fiber reinforced polymer composite (SFRP). The properties are inversely determined using only a few experimental tests. Developing composite manufacturing digital twins for SFRP composite processes, including injection molding and extrusion deposition additive manufacturing (EDAM) requires extensive experimental material characterization. In particular, characterizing the composite mechanical properties is time consuming and therefore, micromechanics models are used to fully identify the elasticity tensor. Hence, the objective of this paper is to infer the fiber orientation and the effective polymer modulus and therefore, identify the elasticity tensor of the composite with minimal experimental tests. To that end, we develop a hierarchical Bayesian model coupled with a micromechanics model to infer the fiber orientation and the polymer elastic modulus simultaneously which we then use to estimate the composite elasticity tensor. We motivate and demonstrate the methodology for the EDAM process but the development is such that it is applicable to other SFRP composites processed via other methods. Our results demonstrate that the approach provides a reliable framework for the inference, with as few as three tensile tests, while accounting for epistemic and aleatory uncertainty. Posterior predictive checks show that the model is able to recreate the experimental data well. The ability of the Bayesian approach to calibrate the material properties and its associated uncertainties, make it a promising tool for enabling a probabilistic predictive framework for composites manufacturing digital twins., Comment: 33 pages, 13 figures, 3 tables

The objective of this paper is to investigate the ability of physics-informed neural networks to learn the magnetic field response as a function of design parameters in the context of a two-dimensional (2-D) magnetostatic problem. Our approach is as follows. First, we present a functional whose minimization is equivalent to solving parametric magnetostatic problems. Subsequently, we use a deep neural network (DNN) to represent the magnetic field as a function of space and parameters that describe geometric features and operating points. We train the DNN by minimizing the physics-informed functional using stochastic gradient descent. Lastly, we demonstrate our approach on a \mbox{ten-dimensional} EI-core electromagnet problem with parameterized geometry. We evaluate the accuracy of the DNN by comparing its predictions to those of finite element analysis., Comment: 12 pages, 10 figures

Probabilistic machine learning models are often insufficient to help with decisions on interventions because those models find correlations - not causal relationships. If observational data is only available and experimentation are infeasible, the correct approach to study the impact of an intervention is to invoke Pearl's causality framework. Even that framework assumes that the underlying causal graph is known, which is seldom the case in practice. When the causal structure is not known, one may use out-of-the-box algorithms to find causal dependencies from observational data. However, there exists no method that also accounts for the decision-maker's prior knowledge when developing the causal structure either. The objective of this paper is to develop rational approaches for making decisions from observational data in the presence of causal graph uncertainty and prior knowledge from the decision-maker. We use ensemble methods like Bayesian Model Averaging (BMA) to infer set of causal graphs that can represent the data generation process. We provide decisions by computing the expected value and risk of potential interventions explicitly. We demonstrate our approach by applying them in different example contexts.

Reliable platforms for data collation during airline schedule operations have significantly increased the quality and quantity of available information for effectively managing airline schedule disruptions. To that effect, this paper applies macroscopic and microscopic techniques by way of basic statistics and machine learning, respectively, to analyze historical scheduling and operations data from a major airline in the United States. Macroscopic results reveal that majority of irregular operations in airline schedule that occurred over a one-year period stemmed from disruptions due to flight delays, while microscopic results validate different modeling assumptions about key drivers for airline disruption management like turnaround as a Gaussian process.

Excessive loads near wounds produce pathological scarring and other complications. Presently, stress cannot easily be measured by surgeons in the operating room. Instead, surgeons rely on intuition and experience. Predictive computational tools are ideal candidates for surgery planning. Finite element (FE) simulations have shown promise in predicting stress fields on large skin patches and complex cases, helping to identify potential regions of complication. Unfortunately, these simulations are computationally expensive and deterministic. However, running a few, well-selected FE simulations allows us to create Gaussian process (GP) surrogate models of local cutaneous flaps that are computationally efficient and able to predict stress and strain for arbitrary material parameters. Here, we create GP surrogates for the advancement, rotation, and transposition flaps. We then use the predictive capability of these surrogates to perform a global sensitivity analysis, ultimately showing that fiber direction has the most significant impact on strain field variations. We then perform an optimization to determine the optimal fiber direction for each flap for three different objectives driven by clinical guidelines. While material properties are not controlled by the surgeon and are actually a source of uncertainty, the surgeon can in fact control the orientation of the flap. Therefore, fiber direction is the only material parameter that can be optimized clinically. The optimization task relies on the efficiency of the GP surrogates to calculate the expected cost of different strategies when the uncertainty of other material parameters is included. We propose optimal flap orientations for the three cost functions and that can help in reducing stress resulting from the surgery and ultimately reduce complications associated with excessive mechanical loading near wounds., Comment: 17 pages, 9 figures. Submitted for publication

Estimating arbitrary quantities of interest (QoIs) that are non-linear operators of complex, expensive-to-evaluate, black-box functions is a challenging problem due to missing domain knowledge and finite budgets. Bayesian optimal design of experiments (BODE) is a family of methods that identify an optimal design of experiments (DOE) under different contexts, using only in a limited number of function evaluations. Under BODE methods, sequential design of experiments (SDOE) accomplishes this task by selecting an optimal sequence of experiments while using data-driven probabilistic surrogate models instead of the expensive black-box function. Probabilistic predictions from the surrogate model are used to define an information acquisition function (IAF) which quantifies the marginal value contributed or the expected information gained by a hypothetical experiment. The next experiment is selected by maximizing the IAF. A generally applicable IAF is the expected information gain (EIG) about a QoI as captured by the expectation of the Kullback-Leibler divergence between the predictive distribution of the QoI after doing a hypothetical experiment and the current predictive distribution about the same QoI. We model the underlying information source as a fully-Bayesian, non-stationary Gaussian process (FBNSGP), and derive an approximation of the information gain of a hypothetical experiment about an arbitrary QoI conditional on the hyper-parameters The EIG about the same QoI is estimated by sample averages to integrate over the posterior of the hyper-parameters and the potential experimental outcomes. We demonstrate the performance of our method in four numerical examples and a practical engineering problem of steel wire manufacturing. The method is compared to two classic SDOE methods: random sampling and uncertainty sampling., Comment: 58 figures

In post-event reconnaissance missions, engineers and researchers collect perishable information about damaged buildings in the affected geographical region to learn from the consequences of the event. A typical post-event reconnaissance mission is conducted by first doing a preliminary survey, followed by a detailed survey. The preliminary survey is typically conducted by driving slowly along a pre-determined route, observing the damage, and noting where further detailed data should be collected. This involves several manual, time-consuming steps that can be accelerated by exploiting recent advances in computer vision and artificial intelligence. The objective of this work is to develop and validate an automated technique to support post-event reconnaissance teams in the rapid collection of reliable and sufficiently comprehensive data, for planning the detailed survey. The technique incorporates several methods designed to automate the process of categorizing buildings based on their key physical attributes, and rapidly assessing their post-event structural condition. It is divided into pre-event and post-event streams, each intending to first extract all possible information about the target buildings using both pre-event and post-event images. Algorithms based on convolutional neural network (CNNs) are implemented for scene (image) classification. A probabilistic approach is developed to fuse the results obtained from analyzing several images to yield a robust decision regarding the attributes and condition of a target building. We validate the technique using post-event images captured during reconnaissance missions that took place after hurricanes Harvey and Irma. The validation data were collected by a structural wind and coastal engineering reconnaissance team, the National Science Foundation (NSF) funded Structural Extreme Events Reconnaissance (StEER) Network.

Despite the wide range of possible scenarios in the aftermath of a disruptive event, each community can make choices to improve its resilience, or its ability to bounce back. A resilient community is one that has prepared for, and can thus absorb, recover from, and adapt to the disruptive event. One important aspect of the recovery phase is assessing the extent of the damage in the built environment through post-event building inspections. In this paper, we develop and demonstrate a resilience-based methodology intended to support rapid post-event decision-making about inspection priorities with limited information. The method uses the basic characteristics of the building stock in a community (floor area, number of stories, type of construction and configuration) to assign structure-specific fragility functions to each building. For an event with a given seismic intensity, the probability of each building reaching a particular damage state is determined, and is used to predict the actual building states and priorities for inspection. Losses are computed based on building usage category, estimated inspection costs, the consequences of erroneous decisions, and the potential for unnecessary restrictions in access. The aim is to provide a means for a community to make rapid cost-based decisions related to inspection of their building inventory. We pose the decision problem as an integer optimization problem that attempts to minimize the expected loss to the community. The advantages of this approach are that it: (i) is simple, (ii) requires minimal inventory data, (iii) is easily scalable, and (iv) does not require significant computing power. Use of this approach before the hazard event can also provide a community with the means to plan and allocate resources in advance of an event to achieve the desirable resiliency goals of the community.

After a disaster, teams of structural engineers collect vast amounts of images from damaged buildings to obtain new knowledge and extract lessons from the event. However, in many cases, the images collected are captured without sufficient spatial context. When damage is severe, it may be quite difficult to even recognize the building. Accessing images of the pre-disaster condition of those buildings is required to accurately identify the cause of the failure or the actual loss in the building. Here, to address this issue, we develop a method to automatically extract pre-event building images from 360o panorama images (panoramas). By providing a geotagged image collected near the target building as the input, panoramas close to the input image location are automatically downloaded through street view services (e.g., Google or Bing in the United States). By computing the geometric relationship between the panoramas and the target building, the most suitable projection direction for each panorama is identified to generate high-quality 2D images of the building. Region-based convolutional neural networks are exploited to recognize the building within those 2D images. Several panoramas are used so that the detected building images provide various viewpoints of the building. To demonstrate the capability of the technique, we consider residential buildings in Holiday Beach, Texas, the United States which experienced significant devastation in Hurricane Harvey in 2017. Using geotagged images gathered during actual post-disaster building reconnaissance missions, we verify the method by successfully extracting residential building images from Google Street View images, which were captured before the event.

Systems engineering processes coordinate the effort of different individuals to generate a product satisfying certain requirements. As the involved engineers are self-interested agents, the goals at different levels of the systems engineering hierarchy may deviate from the system-level goals which may cause budget and schedule overruns. Therefore, there is a need of a systems engineering theory that accounts for the human behavior in systems design. To this end, the objective of this paper is to develop and analyze a principal-agent model of a one-shot (single iteration), shallow (one level of hierarchy) systems engineering process. We assume that the systems engineer maximizes the expected utility of the system, while the subsystem engineers seek to maximize their expected utilities. Furthermore, the systems engineer is unable to monitor the effort of the subsystem engineer and may not have a complete information about their types or the complexity of the design task. However, the systems engineer can incentivize the subsystem engineers by proposing specific contracts. To obtain an optimal incentive, we pose and solve numerically a bi-level optimization problem. Through extensive simulations, we study the optimal incentives arising from different system-level value functions under various combinations of effort costs, problem-solving skills, and task complexities.

Thermal preferences vary from person to person and may change over time. The main objective of this paper is to sequentially pose intelligent queries to occupants in order to optimally learn the indoor air temperature values which maximize their satisfaction. Our central hypothesis is that an occupant's preference relation over indoor air temperature can be described using a scalar function of these temperatures, which we call the "occupant's thermal utility function". Information about an occupant's preference over these temperatures is available to us through their response to thermal preference queries : "prefer warmer," "prefer cooler" and "satisfied" which we interpret as statements about the derivative of their utility function, i.e. the utility function is "increasing", "decreasing" and "constant" respectively. We model this hidden utility function using a Gaussian process prior with built-in unimodality constraint, i.e., the utility function has a unique maximum, and we train this model using Bayesian inference. This permits an expected improvement based selection of next preference query to pose to the occupant, which takes into account both exploration (sampling from areas of high uncertainty) and exploitation (sampling from areas which are likely to offer an improvement over current best observation). We use this framework to sequentially design experiments and illustrate its benefits by showing that it requires drastically fewer observations to learn the maximally preferred temperature values as compared to other methods. This framework is an important step towards the development of intelligent HVAC systems which would be able to respond to occupants' personalized thermal comfort needs. In order to encourage the use of our PE framework and ensure reproducibility in results, we publish an implementation of our work named GPPrefElicit as an open-source package in Python., Comment: 39 pages, 11 figures. References are updated. Typos are corrected. Changed "room temperatures" to "indoor air temperatures"

We present a principal-agent model of a one-shot, shallow, systems engineering process. The process is one-shot in the sense that decisions are made during one time step and that they are final. The term shallow refers to a one-layer hierarchy of the process. Specifically, we assume that the systems engineer has already decomposed the problem in subsystems, and that each subsystem is assigned to a different subsystem engineer. Each subsystem engineer works independently to maximize their own expected payoff. The goal of the systems engineer is to maximize the system-level payoff by incentivizing the subsystem engineers. We restrict our attention to requirement-based system-level payoffs, i.e., the systems engineer makes a profit only if all the design requirements are met. We illustrate the model using the design of an Earth-orbiting satellite system where the systems engineer determines the optimum incentive structures and requirements for two subsystems: the propulsion subsystem and the power subsystem. The model enables the analysis of a systems engineer's decisions about optimal passed-down requirements and incentives for sub-system engineers under different levels of task difficulty and associated costs. Sample results, for the case of risk-neutral systems and subsystems engineers, show that it is not always in the best interest of the systems engineer to pass down the true requirements. As expected, the model predicts that for small to moderate task uncertainties the optimal requirements are higher than the true ones, effectively eliminating the probability of failure for the systems engineer. In contrast, the model predicts that for large task uncertainties the optimal requirements should be smaller than the true ones in order to lure the subsystem engineers into participation.

A problem of considerable importance within the field of uncertainty quantification (UQ) is the development of efficient methods for the construction of accurate surrogate models. Such efforts are particularly important to applications constrained by high-dimensional uncertain parameter spaces. The difficulty of accurate surrogate modeling in such systems, is further compounded by data scarcity brought about by the large cost of forward model evaluations. Traditional response surface techniques, such as Gaussian process regression (or Kriging) and polynomial chaos are difficult to scale to high dimensions. To make surrogate modeling tractable in expensive high-dimensional systems, one must resort to dimensionality reduction of the stochastic parameter space. A recent dimensionality reduction technique that has shown great promise is the method of `active subspaces'. The classical formulation of active subspaces, unfortunately, requires gradient information from the forward model - often impossible to obtain. In this work, we present a simple, scalable method for recovering active subspaces in high-dimensional stochastic systems, without gradient-information that relies on a reparameterization of the orthogonal active subspace projection matrix, and couple this formulation with deep neural networks. We demonstrate our approach on synthetic and real world datasets and show favorable predictive comparison to classical active subspaces., Comment: 9 pages, 4 figures

Stochastic partial differential equations (SPDEs) are ubiquitous in engineering and computational sciences. The stochasticity arises as a consequence of uncertainty in input parameters, constitutive relations, initial/boundary conditions, etc. Because of these functional uncertainties, the stochastic parameter space is often high-dimensional, requiring hundreds, or even thousands, of parameters to describe it. This poses an insurmountable challenge to response surface modeling since the number of forward model evaluations needed to construct an accurate surrogate grows exponentially with the dimension of the uncertain parameter space; a phenomenon referred to as the \textit{curse of dimensionality}. State-of-the-art methods for high-dimensional uncertainty propagation seek to alleviate the curse of dimensionality by performing dimensionality reduction in the uncertain parameter space. However, one still needs to perform forward model evaluations that potentially carry a very high computational burden. We propose a novel methodology for high-dimensional uncertainty propagation of elliptic SPDEs which lifts the requirement for a deterministic forward solver. Our approach is as follows. We parameterize the solution of the elliptic SPDE using a deep residual network (ResNet). In a departure from the traditional squared residual (SR) based loss function for training the ResNet, we introduce a novel physics-informed loss function derived from variational principles. Specifically, our loss function is the expectation of the energy functional of the PDE over the stochastic variables. We demonstrate our solver-free approach through various examples where the elliptic SPDE is subjected to different types of high-dimensional input uncertainties. Also, we solve high-dimensional uncertainty propagation and inverse problems., Comment: 63 pages, 32 figures

After a disaster, teams of structural engineers collect vast amounts of images from damaged buildings to obtain lessons and gain knowledge from the event. Images of damaged buildings and components provide valuable evidence to understand the consequences on our structures. However, in many cases, images of damaged buildings are often captured without sufficient spatial context. Also, they may be hard to recognize in cases with severe damage. Incorporating past images showing a pre-disaster condition of such buildings is helpful to accurately evaluate possible circumstances related to a building's failure. One of the best resources to observe the pre-disaster condition of the buildings is Google Street View. A sequence of 360 panorama images which are captured along streets enables all-around views at each location on the street. Once a user knows the GPS information near the building, all external views of the building can be made available. In this study, we develop an automated technique to extract past building images from 360 panorama images serviced by Google Street View. Users only need to provide a geo-tagged image, collected near the target building, and the rest of the process is fully automated. High-quality and undistorted building images are extracted from past panoramas. Since the panoramas are collected from various locations near the building along the street, the user can identify its pre-disaster conditions from the full set of external views.

Bayesian optimal design of experiments (BODE) has been successful in acquiring information about a quantity of interest (QoI) which depends on a black-box function. BODE is characterized by sequentially querying the function at specific designs selected by an infill-sampling criterion. However, most current BODE methods operate in specific contexts like optimization, or learning a universal representation of the black-box function. The objective of this paper is to design a BODE for estimating the statistical expectation of a physical response surface. This QoI is omnipresent in uncertainty propagation and design under uncertainty problems. Our hypothesis is that an optimal BODE should be maximizing the expected information gain in the QoI. We represent the information gain from a hypothetical experiment as the Kullback-Liebler (KL) divergence between the prior and the posterior probability distributions of the QoI. The prior distribution of the QoI is conditioned on the observed data and the posterior distribution of the QoI is conditioned on the observed data and a hypothetical experiment. The main contribution of this paper is the derivation of a semi-analytic mathematical formula for the expected information gain about the statistical expectation of a physical response. The developed BODE is validated on synthetic functions with varying number of input-dimensions. We demonstrate the performance of the methodology on a steel wire manufacturing problem., Comment: 27 pages, 19 figures

State-of-the-art computer codes for simulating real physical systems are often characterized by a vast number of input parameters. Performing uncertainty quantification (UQ) tasks with Monte Carlo (MC) methods is almost always infeasible because of the need to perform hundreds of thousands or even millions of forward model evaluations in order to obtain convergent statistics. One, thus, tries to construct a cheap-to-evaluate surrogate model to replace the forward model solver. For systems with large numbers of input parameters, one has to deal with the curse of dimensionality - the exponential increase in the volume of the input space, as the number of parameters increases linearly. In this work, we demonstrate the use of deep neural networks (DNN) to construct surrogate models for numerical simulators. We parameterize the structure of the DNN in a manner that lends the DNN surrogate the interpretation of recovering a low dimensional nonlinear manifold. The model response is a parameterized nonlinear function of the low dimensional projections of the input. We think of this low dimensional manifold as a nonlinear generalization of the notion of the active subspace. Our approach is demonstrated with a problem on uncertainty propagation in a stochastic elliptic partial differential equation (SPDE) with uncertain diffusion coefficient. We deviate from traditional formulations of the SPDE problem by not imposing a specific covariance structure on the random diffusion coefficient. Instead, we attempt to solve a more challenging problem of learning a map between an arbitrary snapshot of the diffusion field and the response.

Design optimization of engineering systems with multiple competing objectives is a painstakingly tedious process especially when the objective functions are expensive-to-evaluate computer codes with parametric uncertainties. The effectiveness of the state-of-the-art techniques is greatly diminished because they require a large number of objective evaluations, which makes them impractical for problems of the above kind. Bayesian global optimization (BGO), has managed to deal with these challenges in solving single-objective optimization problems and has recently been extended to multi-objective optimization (MOO). BGO models the objectives via probabilistic surrogates and uses the epistemic uncertainty to define an information acquisition function (IAF) that quantifies the merit of evaluating the objective at new designs. This iterative data acquisition process continues until a stopping criterion is met. The most commonly used IAF for MOO is the expected improvement over the dominated hypervolume (EIHV) which in its original form is unable to deal with parametric uncertainties or measurement noise. In this work, we provide a systematic reformulation of EIHV to deal with stochastic MOO problems. The primary contribution of this paper lies in being able to filter out the noise and reformulate the EIHV without having to observe or estimate the stochastic parameters. An addendum of the probabilistic nature of our methodology is that it enables us to characterize our confidence about the predicted Pareto front. We verify and validate the proposed methodology by applying it to synthetic test problems with known solutions. We demonstrate our approach on an industrial problem of die pass design for a steel wire drawing process., Comment: 19 pages, 14 figures

This work is concerned with the quantification of the epistemic uncertainties induced the discretization of partial differential equations. Following the paradigm of probabilistic numerics, we quantify this uncertainty probabilistically. Namely, we develop a probabilistic solver suitable for linear partial differential equations (PDE) with mixed (Dirichlet and Neumann) boundary conditions defined on arbitrary geometries. The idea is to assign a probability measure on the space of solutions of the PDE and then condition this measure by enforcing that the PDE and the boundary conditions are satisfied at a finite set of spatial locations. The resulting posterior probability measure quantifies our state of knowledge about the solution of the problem given this finite discretization.

Design optimization under uncertainty is notoriously difficult when the objective function is expensive to evaluate. State-of-the-art techniques, e.g, stochastic optimization or sampling average approximation, fail to learn exploitable patterns from collected data and require an excessive number of objective function evaluations. There is a need for techniques that alleviate the high cost of information acquisition and select sequential simulations optimally. In the field of deterministic single-objective unconstrained global optimization, the Bayesian global optimization (BGO) approach has been relatively successful in addressing the information acquisition problem. BGO builds a probabilistic surrogate of the expensive objective function and uses it to define an information acquisition function (IAF) whose role is to quantify the merit of making new objective evaluations. Specifically, BGO iterates between making the observations with the largest expected IAF and rebuilding the probabilistic surrogate, until a convergence criterion is met. In this work, we extend the expected improvement (EI) IAF to the case of design optimization under uncertainty wherein the EI policy is reformulated to filter out parametric and measurement uncertainties. To increase the robustness of our approach in the low sample regime, we employ a fully Bayesian interpretation of Gaussian processes by constructing a particle approximation of the posterior of its hyperparameters using adaptive Markov chain Monte Carlo. We verify and validate our approach by solving two synthetic optimization problems under uncertainty and demonstrate it by solving the oil-well-placement problem with uncertainties in the permeability field and the oil price time series., Comment: 10 pages, 21 figures

The prohibitive cost of performing Uncertainty Quantification (UQ) tasks with a very large number of input parameters can be addressed, if the response exhibits some special structure that can be discovered and exploited. Several physical responses exhibit a special structure known as an active subspace (AS), a linear manifold of the stochastic space characterized by maximal response variation. The idea is that one should first identify this low dimensional manifold, project the high-dimensional input onto it, and then link the projection to the output. In this work, we develop a probabilistic version of AS which is gradient-free and robust to observational noise. Our approach relies on a novel Gaussian process regression with built-in dimensionality reduction with the AS represented as an orthogonal projection matrix that serves as yet another covariance function hyper-parameter to be estimated from the data. To train the model, we design a two-step maximum likelihood optimization procedure that ensures the orthogonality of the projection matrix by exploiting recent results on the Stiefel manifold. The additional benefit of our probabilistic formulation is that it allows us to select the dimensionality of the AS via the Bayesian information criterion. We validate our approach by showing that it can discover the right AS in synthetic examples without gradient information using both noiseless and noisy observations. We demonstrate that our method is able to discover the same AS as the classical approach in a challenging one-hundred-dimensional problem involving an elliptic stochastic partial differential equation with random conductivity. Finally, we use our approach to study the effect of geometric and material uncertainties in the propagation of solitary waves in a one-dimensional granular system., Comment: 37 pages, 20 figures