Search

Designs for computer experiments in quantitative factors use columns with many levels. Filling the experimental space is their most important property, and there are many criteria that assess aspects of space‐filling. Recently, Tian and Xu proposed a stratification pattern for assessing the stratification‐related space‐filling properties of designs for quantitative experimental variables whose number of levels is a power of a – usually small – integer. Such designs have been named GSOAs, in generalization of the earlier proposal of strong – or stratum – orthogonal arrays (SOAs). Latin hypercube designs (LHDs) with a suitable number of levels are special cases of GSOAs. Tian and Xu proposed to use the stratification pattern as a means to ranking (G)SOAs. They reported a small simulation study in which arrays that fared well in that ranking performed well in predicting an unknown function. Shi and Xu refined the criterion and also demonstrated success of a design that fares well on their refined criterion. This paper explains the ideas behind the stratification pattern and the related ranking criteria. A practical example and several toy examples aid the illustration. The stratification pattern can be calculated using the R package SOAs, which does not only provide the pattern itself but also provides more detail in a dimension by weight table, in the spirit of the refinement by Shi and Xu. [ABSTRACT FROM AUTHOR]

Copyright of Cybernetics & Systems Analysis / Kibernetiki i Sistemnyj Analiz is the property of V.M. Glushkov Institute of Cybernetics of NAS of Ukraine and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

AbstractIn customer-driven design of systems or products, one has performance targets in mind and would like to determine values for product or system parameters that meet such targets. Engineering computer simulation models predict performance given design parameter values; meeting a target is done iteratively through an optimization search procedure, typically by optimizing a regression, neural network or other type of approximation metamodel of the computer model. We pose the thesis that, since forward metamodel construction is a key part of this strategy, constructing an inverse metamodel directly is advantageous. The inverse metamodel obviates the need for optimization in many settings. One can design experiments that allow simultaneous fitting of forward and inverse metamodels. We discuss the potential for this strategy, the connection with the calibration problem, and some of the issues that must be resolved to make the approach practical. [ABSTRACT FROM AUTHOR]

Computer model calibration is a method to identify the unknown parameters of computer models, which is attaining more and more attention now. Most of the existing articles develop the calibration procedure under the assumption that the sample size of the physical experiments is larger than the dimension of the calibration parameters, which would not be satisfied in practice. In this article, we propose a sparse estimator of the calibration parameters and its robust version based on adaptive lasso penalty with adapting to the sample size of the physical experiments and the dimension of the calibration parameters, and the proposed robust estimator can deal with the heavy-tailed error and outliers efficiently. Subsequently, we investigate the nonasymptotic properties of the proposed estimators and obtain an upper bound of l 2 error of the proposed estimators by the concentration inequalities. We conduct some numerical simulations and an application to composite fuselage simulation, which verify that the proposed estimators enjoy nice performance. [ABSTRACT FROM AUTHOR]

AbstractActive subspace (AS) methods are a valuable tool for understanding the relationship between the inputs and outputs of a Physics simulation. In this article, an elegant generalization of the traditional ASM is developed to assess the co-activity of two computer models. This generalization, which we refer to as a Co-Active Subspace (Co-AS) Method, allows for the joint analysis of two or more computer models allowing for thorough exploration of the alignment (or non-alignment) of the respective gradient spaces. We define co-active directions, co-sensitivity indices, and a scalar “concordance” metric (and complementary “discordance” pseudo-metric) and we demonstrate that these are powerful tools for understanding the behavior of a class of computer models, especially when used to supplement traditional AS analysis. Details for efficient estimation of the Co-AS and an accompanying R package (concordance) are provided. Practical application is demonstrated through analyzing a set of simulated rate stick experiments for PBX 9501, a high explosive, offering insights into complex model dynamics. [ABSTRACT FROM AUTHOR]

This article proposes the use of the Karhunen Loève expansion in the design of computer experiments when the Gaussian Process (GP) model is used to model the output, Y(x). We start by finding the K-L expansion in the case of the univariate and multivariate processes. When the analytical solution does not exist, we apply several methods of approximation, such as Haar wavelet functions and Fourier expansions. Numerical examples are presented to illustrate the idea of the approximations. We use the approximated model to select the maximum entropy sampling design. [ABSTRACT FROM AUTHOR]

In many practical experiments, the number of possible levels for each experimental factor is often limited to a finite number. Various discrepancies, like discrete discrepancy and Lee discrepancy, have been proposed to measure the uniformity on the discrete experimental domain. However, these two discrepancies can yield unreasonable results when multiple levels are involved. In this paper, a new discrepancy, termed absolute discrepancy (AD), is proposed as a more reasonable measure of uniformity on a discrete experimental domain, particularly for discrete-numerical factors. Moreover, connections between AD and other design optimality criteria, such as generalized minimum aberration and orthogonality, are established. These connections provide strong statistical justification for AD. Some lower bounds of AD are also obtained, serving as important benchmarks for the design uniformity. In addition, uniform designs based on the AD criterion have produced better prediction performance by fitting statistical surrogate models of complex surfaces compared to their competitors. Furthermore, the application in hyperparameter optimization for machine learning algorithms also validates the superiority of uniform designs with the proposed AD criterion. [ABSTRACT FROM AUTHOR]

AbstractDesign of Experiments (DoE) is a relevant class to undergraduate students in the sciences, because it teaches them how to plan, conduct, and analyze experiments. In the literature on DoE, there are several contributions to its pedagogy, such as easy-to-use class experiments, virtual experiments, and software to construct experimental designs. However, there are virtually no systematic evaluations of the actual DoE pedagogy. To address this issue, we build the first database of DoE courses offered to undergraduate students in the United States. The database has records on courses offered from 2019 to 2022 at the best universities in the US News Best National Universities ranking of 2022. Specifically, it has data on 18 general and content-specific features of 206 courses. To study the DoE pedagogy, we analyze the database using descriptive statistics and text mining. Based on our analysis, we provide instructors with recommendations and teaching material to enhance their DoE courses. The database and material are included in the supplement of this article. [ABSTRACT FROM AUTHOR]

Partial differential equations (PDEs) have become an essential tool for modeling complex physical systems. Such equations are typically solved numerically via mesh-based methods, such as finite element methods, with solutions over the spatial domain. However, obtaining these solutions are often prohibitively costly, limiting the feasibility of exploring parameters in PDEs. In this article, we propose an efficient emulator that simultaneously predicts the solutions over the spatial domain, with theoretical justification of its uncertainty quantification. The novelty of the proposed method lies in the incorporation of the mesh node coordinates into the statistical model. In particular, the proposed method segments the mesh nodes into multiple clusters via a Dirichlet process prior and fits Gaussian process models with the same hyperparameters in each of them. Most importantly, by revealing the underlying clustering structures, the proposed method can provide valuable insights into qualitative features of the resulting dynamics that can be used to guide further investigations. Real examples are demonstrated to show that our proposed method has smaller prediction errors than its main competitors, with competitive computation time, and identifies interesting clusters of mesh nodes that possess physical significance, such as satisfying boundary conditions. An R package for the proposed methodology is provided in an open repository. [ABSTRACT FROM AUTHOR]

AbstractMethod of constructing new and useful series of m-associate class partially balanced incomplete block (PBIB) designs, named as Generalized Extended Triangular (GET) designs, in m replications has been proposed along with an outline of analytical procedure. Triangular designs, one of the most popular and widely used class of PBIB(2) designs, and the extended triangular PBIB(3) designs form other two important particular cases of this generalized class of designs. To avoid the complexity involved in the construction methodology and to save time, an R package ‘‘GETdesigns’’ has been developed for the generation of the proposed designs. Further, these designs have been effectively used for obtaining mating designs and Latin hypercube designs. [ABSTRACT FROM AUTHOR]

Dimension reduction techniques have long been an important topic in statistics, and active subspaces (AS) have received much attention this past decade in the computer experiments literature. The most common approach toward estimating the AS is to use Monte Carlo with numerical gradient evaluation. While sensible in some settings, this approach has obvious drawbacks. Recent research has demonstrated that active subspace calculations can be obtained in closed form, conditional on a Gaussian process (GP) surrogate, which can be limiting in high-dimensional settings for computational reasons. In this article, we produce the relevant calculations for a more general case when the model of interest is a linear combination of tensor products. These general equations can be applied to the GP, recovering previous results as a special case, or applied to the models constructed by other regression techniques including multivariate adaptive regression splines (MARS). Using a MARS surrogate has many advantages including improved scaling, better estimation of active subspaces in high dimensions and the ability to handle a large number of prior distributions in closed form. In one real-world example, we obtain the active subspace of a radiation-transport code with 240 inputs and 9372 model runs in under half an hour. for this article are available online. [ABSTRACT FROM AUTHOR]

In this work we propose a low rank approximation of areal, particularly three dimensional, data utilizing additional weights. This way we enable effective compression if additional information indicates that parts of the data are of higher interest than others. The guiding example are high fidelity finite element simulations of an abdominal aortic aneurysm, i.e. a deformed blood vessel. The additional weights encapsulate the areas of high stress, which we assume indicates the rupture risk of the aorta. The stress values on the grid are modeled as a Gaussian Markov random field and we define our approximation as a basis of vectors that solve a series of optimization problems. Each of these problems describes the minimization of an expected weighted quadratic loss. We provide an effective numerical heuristic to compute the basis under general conditions, which relies on the sparsity of the precision matrix to ensure acceptable computing time even for large grids. We explicitly explore two such bases on the surface of a high fidelity finite element grid and show their efficiency for compression. Finally, we utilize the approach as part of a larger model to predict the van Mises stress in areas of interest using low and high fidelity simulations. [ABSTRACT FROM AUTHOR]

Although high-performance computing (HPC) systems have been scaled to meet the exponentially growing demand for scientific computing, HPC performance variability remains a major challenge in computer science. Statistically, performance variability can be characterized by a distribution. Predicting performance variability is a critical step in HPC performance variability management. In this article, we propose a new framework to predict performance distributions. The proposed framework is a modified Gaussian process that can predict the distribution function of the input/output (I/O) throughput under a specific HPC system configuration. We also impose a monotonic constraint so that the predicted function is nondecreasing, which is a property of the cumulative distribution function. Additionally, the proposed model can incorporate both quantitative and qualitative input variables. We predict the HPC I/O distribution using the proposed method for the IOzone variability data. Data analysis results show that our framework can generate accurate predictions, and outperform existing methods. We also show how the predicted functional output can be used to generate predictions for a scalar summary of the performance distribution, such as the mean, standard deviation, and quantiles. Our prediction results can further be used for HPC system variability monitoring and optimization. This article has online supplementary materials. [ABSTRACT FROM AUTHOR]

Computer models cannot always fit the physical systems well in practice. Most of these models contain unknown parameters with high uncertainty. Calibration is an approach to identify these unknown parameters in computer models. Inspired by Tuo and Wu (Ann Stat 43(6):2331–2352, 2015), we propose a new calibration procedure based on the smoothing spline ANOVA, which can be regarded as an extension of the methods of Tuo and Wu (Ann Stat 43(6):2331–2352, 2015). The proposed procedure mainly comprises two steps: computing the improved L 2 estimator of the calibration parameters, and estimating the discrepancy function. We derive the rate of convergence of the proposed estimator of the calibration parameters and investigate its asymptotic properties. The proposed method combines the advantages of L 2 calibration and ordinary least square (OLS) calibration, and exhibits better performances than Tuo and Wu (Ann Stat 43(6):2331–2352, 2015) and Wong et al. (J R Stat Soc Ser B 79(2):635–645, 2017). We conduct some numerical simulations and apply the proposed method to two real examples, which demonstrate the advantages of the proposed procedure. [ABSTRACT FROM AUTHOR]

AbstractThis article proposes a new kriging that has a rational form. It is shown that the generalized least squares estimator of the mean from rational kriging is much more well behaved than that of ordinary kriging. Parameter estimation and uncertainty quantification for rational kriging are proposed using a Gaussian process framework. A generalized version of rational kriging is also proposed, which includes ordinary and rational kriging as special cases. Extensive simulations carried out over a wide class of functions show that the generalized rational kriging performs on par or better than both ordinary and rational kriging in terms of prediction and uncertainty quantification. The only extra step needed for generalized rational kriging over ordinary kriging is the computation of Perron eigenvector of an augmented correlation matrix which can be computed in near linear time and therefore, its overall computational complexity is no more than that of ordinary kriging. The potential applications of the new kriging methods in the emulation of computationally expensive models and model calibration problems are illustrated with real and simulated examples. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work. [ABSTRACT FROM AUTHOR]

In this work, we propose a novel framework for large-scale Gaussian process (GP) modeling. Contrary to the global, and local approximations proposed in the literature to address the computational bottleneck with exact GP modeling, we employ a combined global-local approach in building the approximation. Our framework uses a subset-of-data approach where the subset is a union of a set of global points designed to capture the global trend in the data, and a set of local points specific to a given testing location to capture the local trend around the testing location. The correlation function is also modeled as a combination of a global, and a local kernel. The predictive performance of our framework, which we refer to as TwinGP, is comparable to the state-of-the-art GP modeling methods, but at a fraction of their computational cost. [ABSTRACT FROM AUTHOR]

With advances in scientific computing and mathematical modeling, complex scientific phenomena such as galaxy formations and rocket propulsion can now be reliably simulated. Such simulations can however be very time-intensive, requiring millions of CPU hours to perform. One solution is multi-fidelity emulation, which uses data of different fidelities to train an efficient predictive model which emulates the expensive simulator. For complex scientific problems and with careful elicitation from scientists, such multi-fidelity data may often be linked by a directed acyclic graph (DAG) representing its scientific model dependencies. We thus propose a new Graphical Multi-fidelity Gaussian Process (GMGP) model, which embeds this DAG structure (capturing scientific dependencies) within a Gaussian process framework. We show that the GMGP has desirable modeling traits via two Markov properties, and admits a scalable algorithm for recursive computation of the posterior mean and variance along at each depth level of the DAG. We also present a novel experimental design methodology over the DAG given an experimental budget, and propose a nonlinear extension of the GMGP via deep Gaussian processes. The advantages of the GMGP are then demonstrated via a suite of numerical experiments and an application to emulation of heavy-ion collisions, which can be used to study the conditions of matter in the Universe shortly after the Big Bang. The proposed model has broader uses in data fusion applications with graphical structure, which we further discuss. [ABSTRACT FROM AUTHOR]

In modern computer experiment applications, one often encounters the situation where various models of a physical system are considered, each implemented as a simulator on a computer. An important question in such a setting is determining the best simulator, or the best combination of simulators, to use for prediction and inference. Bayesian model averaging (BMA) and stacking are two statistical approaches used to account for model uncertainty by aggregating a set of predictions through a simple linear combination or weighted average. Bayesian model mixing (BMM) extends these ideas to capture the localized behavior of each simulator by defining input-dependent weights. One possibility is to define the relationship between inputs and the weight functions using a flexible nonparametric model that learns the local strengths and weaknesses of each simulator. This article proposes a BMM model based on Bayesian Additive Regression Trees (BART). The proposed methodology is applied to combine predictions from Effective Field Theories (EFTs) associated with a motivating nuclear physics application. for this article are available online. Source code is available at . [ABSTRACT FROM AUTHOR]

This paper introduces a new way to extract a set of representative points from a continuous distribution, which focuses on a method where the selection of points is essentially deterministic, with an emphasis on achieving accurate approximation when the size of points is small. These points are generated by minimizing the Kullback–Leibler divergence, which is an information-based measure of the disparity between two probability distributions. We refer to these points as Kullback–Leibler points. Based on the link between the total variation and the Kullback–Leibler divergence, we prove that the empirical distribution of Kullback–Leibler points converges to the target distribution. Additionally, we illustrate that Kullback–Leibler points have advantages in simulations when compared with representative points generated by Monte Carlo or other representative points methods. In addition, to prevent the frequent evaluation of complex functions, a sequential version of Kullback–Leibler points is proposed, which adaptively updates the representative points by learning about the complex or unknown functions sequentially. Two potential applications of Kullback-Leibler points in simulation of complex probability densities and optimization of complex response surfaces are discussed and demonstrated with examples. [ABSTRACT FROM AUTHOR]

AbstractCalibration refers to the statistical estimation of unknown model parameters in computer experiments, such that computer experiments can match underlying physical systems. This work develops a new calibration method for imperfect computer models, Sobolev calibration, which can rule out calibration parameters that generate overfitting calibrated functions. We prove that the Sobolev calibration enjoys desired theoretical properties including fast convergence rate, asymptotic normality and semiparametric efficiency. We also demonstrate an interesting property that the Sobolev calibration can bridge the gap between two influential methods: L2 calibration and Kennedy and O’Hagan’s calibration. In addition to exploring the deterministic physical experiments, we theoretically justify that our method can transfer to the case when the physical process is indeed a Gaussian process, which follows the original idea of Kennedy and O’Hagan’s. Numerical simulations as well as a real-world example illustrate the competitive performance of the proposed method. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work. [ABSTRACT FROM AUTHOR]

In projection pursuit regression (PPR), a univariate response variable is approximated by the sum of M “ridge functions,” which are flexible functions of one-dimensional projections of a multivariate input variable. Traditionally, optimization routines are used to choose the projection directions and ridge functions via a sequential algorithm, and M is typically chosen via cross-validation. We introduce a novel Bayesian version of PPR, which has the benefit of accurate uncertainty quantification. To infer appropriate projection directions and ridge functions, we apply novel adaptations of methods used for the single ridge function case ( M = 1 ), called the Bayesian Single Index Model; and use a Reversible Jump Markov chain Monte Carlo algorithm to infer the number of ridge functions M. We evaluate the predictive ability of our model in 20 simulated scenarios and for 23 real datasets, in a bake-off against an array of state-of-the-art regression methods. Finally, we generalize this methodology and demonstrate the ability to accurately model multivariate response variables. Its effective performance indicates that Bayesian Projection Pursuit Regression is a valuable addition to the existing regression toolbox. [ABSTRACT FROM AUTHOR]

Model calibration is crucial for optimizing the performance of complex computer models across various disciplines. In the era of Industry 4.0, symbolizing rapid technological advancement through the integration of advanced digital technologies into industrial processes, model calibration plays a key role in advancing digital twin technology, ensuring alignment between digital representations and real‐world systems. This comprehensive review focuses on the Kennedy and O'Hagan (KOH) framework (Kennedy and O'Hagan, Journal of the Royal Statistical Society: Series B 2001; 63(3):425–464). In particular, we explore recent advancements addressing the challenges of the unidentifiability issue while accommodating model inadequacy within the KOH framework. In addition, we explore recent advancements in adapting the KOH framework to complex scenarios, including those involving multivariate outputs and functional calibration parameters. We also delve into experimental design strategies tailored to the unique demands of model calibration. By offering a comprehensive analysis of the KOH approach and its diverse applications, this review serves as a valuable resource for researchers and practitioners aiming to enhance the accuracy and reliability of their computer models. This article is categorized under:Statistical Models > Semiparametric ModelsStatistical Models > Simulation ModelsStatistical Models > Bayesian Models [ABSTRACT FROM AUTHOR]

Abstract The testing problem for the homogeneity of a location-scale mixture model is fundamental for subsequent statistical inference. In this paper, we employ the EM-test method to test the homogeneity of two commonly encountered location-scale mixture models with a structural scale, and we also establish the theoretical properties of test statistics. Further, to enhance the reliability of the test, based on the computer experiments approach, we develop a Bartlett-corrected approximate distribution for the proposed test statistics. Through Monte Carlo studies and real data example, the effectiveness of the proposed testing procedure is compared with that of the classical one, especially for small or moderate sample sizes. [ABSTRACT FROM AUTHOR]

A primary goal of computer experiments is to reconstruct the function given by the computer code via scattered evaluations. Traditional isotropic Gaussian process models suffer from the curse of dimensionality, when the input dimension is relatively high given limited data points. Gaussian process models with additive correlation functions are scalable to dimensionality, but they are more restrictive as they only work for additive functions. In this work, we consider a projection pursuit model, in which the nonparametric part is driven by an additive Gaussian process regression. We choose the dimension of the additive function higher than the original input dimension, and call this strategy "dimension expansion". We show that dimension expansion can help approximate more complex functions. A gradient descent algorithm is proposed for model training based on the maximum likelihood estimation. Simulation studies show that the proposed method outperforms the traditional Gaussian process models. [ABSTRACT FROM AUTHOR]

An explosion in the availability of rich data from the technological advances is hindering efforts at statistical analysis due to constraints on time and memory storage, regardless of whether researchers employ simple methods (e.g., linear regression) or complex models (e.g., Gaussian processes). A recent approach to overcoming these limits involves information-based optimal subdata selection and Latin hypercube subagging. In the current study, we develop a novel subdata selection method for large-scale computer models based on expected improvement optimization. Numerical and empirical analysis using real-world data are used to select subdata by which to derive accurate predictions. During the optimization procedure, the proposed scheme employs the geometry of the input feature region as well as information related to output values. The data points associated with the largest improvement in prediction accuracy are combined in the construction of a subdataset that can be used to formulate predictions with affordable computing time. for this article, including proofs of theorems and additional numerical results, are available online. [ABSTRACT FROM AUTHOR]

We analyse the performance of several iterative algorithms for the quantisation of a probability measure μ , based on the minimisation of a Maximum Mean Discrepancy (MMD). Our analysis includes kernel herding, greedy MMD minimisation and Sequential Bayesian Quadrature (SBQ). We show that the finite-sample-size approximation error, measured by the MMD, decreases as 1/n for SBQ and also for kernel herding and greedy MMD minimisation when using a suitable step-size sequence. The upper bound on the approximation error is slightly better for SBQ, but the other methods are significantly faster, with a computational cost that increases only linearly with the number of points selected. This is illustrated by two numerical examples, with the target measure μ being uniform (a space-filling design application) and with μ a Gaussian mixture. They suggest that the bounds derived in the paper are overly pessimistic, in particular for SBQ. The sources of this pessimism are identified but seem difficult to counter. [ABSTRACT FROM AUTHOR]

Closed die hot forging of 304L stainless steel is difficult compared to carbon and micro-alloyed steels in terms of higher forming force and defects. An appropriate combination of design and process factors including cross-section diameter, initial billet temperature, and flash land thickness is crucial to minimize the forming force without causing any under-filling or folding defects. This paper aims to develop an optimized closed-die hot forging process for long, thin, and tubular rail body components used in gasoline direct injection systems. The effects on the forging load are investigated using computer experiments containing the finite element method under a full factorial array of three factors at three levels to obtain the main and interaction effects. Linear and quadratic surface response analyses are performed to define the relation between forging load and input factors, and finally, the simulations are validated with some limited experiments. The simulation results that fit the nonlinear regression model were in close agreement with the experiments, and an optimal set of parameters is conveniently proposed for industrial production. [ABSTRACT FROM AUTHOR]

Bayesian calibration of black-box computer models offers an established framework for quantification of uncertainty of model parameters and predictions. Traditional Bayesian calibration involves the emulation of the computer model and an additive model discrepancy term using Gaussian processes; inference is then carried out using Markov chain Monte Carlo. This calibration approach is limited by the poor scalability of Gaussian processes and by the need to specify a sensible covariance function to deal with the complexity of the computer model and the discrepancy. In this work, we propose a novel calibration framework, where these challenges are addressed by means of compositions of Gaussian processes into Deep Gaussian processes and scalable variational inference techniques. Thanks to this formulation, it is possible to obtain a flexible calibration approach, which is easy to implement in development environments featuring automatic differentiation and exploiting GPU-type hardware. We show how our proposal yields a powerful alternative to the state-of-the-art by means of experimental validations on various calibration problems. We conclude the paper by showing how we can carry out adaptive experimental design, and by discussing the identifiability properties of the proposed calibration model. [ABSTRACT FROM AUTHOR]

In the continual effort to improve product quality and decrease operations costs, computational modeling is increasingly being deployed to determine feasibility of product designs or configurations. Surrogate modeling of these computer experiments via local models, which induce sparsity by only considering short range interactions, can tackle huge analyses of complicated input–output relationships. However, narrowing focus to local scale means that global trends must be relearned over and over again. In this article, we propose a framework for incorporating information from a global sensitivity analysis into the surrogate model as an input rotation and rescaling preprocessing step. We discuss the relationship between several sensitivity analysis methods based on kernel regression before describing how they give rise to a transformation of the input variables. Specifically, we perform an input warping such that the "warped simulator" is equally sensitive to all input directions, freeing local models to focus on local dynamics. Numerical experiments on observational data and benchmark test functions, including a high-dimensional computer simulator from the automotive industry, provide empirical validation. [ABSTRACT FROM AUTHOR]

Techniques to reduce the energy burden of an industrial ecosystem often require solving a multiobjective optimization problem. However, collecting experimental data can often be either expensive or time-consuming. In such cases, statistical methods can be helpful. This article proposes Pareto Front (PF) and Pareto Set (PS) estimation methods using Bayesian Additive Regression Trees (BART), which is a nonparametric model whose assumptions are typically less restrictive than popular alternatives, such as Gaussian Processes (GPs). These less restrictive assumptions allow BART to handle scenarios (e.g., high-dimensional input spaces, nonsmooth responses, large datasets) that GPs find difficult. The performance of our BART-based method is compared to a GP-based method using analytic test functions, demonstrating convincing advantages. Finally, our BART-based methodology is applied to a motivating engineering problem. , which include a theorem proof, algorithms, and R code, for this article are available online. [ABSTRACT FROM AUTHOR]

A computer model can be used for predicting an output only after specifying the values of some unknown physical constants known as calibration parameters. The unknown calibration parameters can be estimated from real data by conducting physical experiments. This paper presents an approach to optimally design such a physical experiment. The problem of optimally designing a physical experiment, using a computer model, is similar to the problem of finding an optimal design for fitting nonlinear models. However, the problem is more challenging than the existing work on nonlinear optimal design because of the possibility of model discrepancy, that is, the computer model may not be an accurate representation of the true underlying model. Therefore, we propose an optimal design approach that is robust to potential model discrepancies. We show that our designs are better than the commonly used physical experimental designs that do not make use of the information contained in the computer model and other nonlinear optimal designs that ignore potential model discrepancies. We illustrate our approach using a toy example and a real example from industry. [ABSTRACT FROM AUTHOR]

Let Z n = { Z 1 , ... , Z n } be a design; that is, a collection of n points Z j ∈ [ - 1 , 1 ] d . We study the quality of quantisation of [ - 1 , 1 ] d by the points of Z n and the problem of quality of coverage of [ - 1 , 1 ] d by B d (Z n , r) , the union of balls centred at Z j ∈ Z n . We concentrate on the cases where the dimension d is not small, d ≥ 5 , and n is not too large, n ≤ 2 d . We define the design D n , δ as a 2 d - 1 design defined on vertices of the cube [ - δ , δ ] d , 0 ≤ δ ≤ 1 . For this design, we derive a closed-form expression for the quantisation error and very accurate approximations for the coverage area vol ([ - 1 , 1 ] d ∩ B d (Z n , r)) . We provide results of a large-scale numerical investigation confirming the accuracy of the developed approximations and the efficiency of the designs D n , δ . [ABSTRACT FROM AUTHOR]

Efficient global optimization (EGO) is one of the most popular sequential adaptive design (SAD) methods for expensive black-box optimization problems. A well-recognized weakness of the original EGO in complex computer experiments is that it is serial, and hence the modern parallel computing techniques cannot be utilized to speed up the running of simulator experiments. For those multiple points EGO methods, the heavy computation and points clustering are the obstacles. In this work, a novel batch SAD method, named "Accelerated EGO", is forwarded by using a refined sampling/importance resampling (SIR) method to search the points with large expected improvement (EI) values. The computation burden of the new method is much lighter, and the points clustering is also avoided. The efficiency of the proposed batch SAD is validated by nine classic test functions with dimension from 2 to 12. The empirical results show that the proposed algorithm indeed can parallelize original EGO, and gain much improvement compared against the other parallel EGO algorithm especially under high-dimensional case. Additionally, the new method is applied to the hyper-parameter tuning of support vector machine (SVM) and XGBoost models in machine learning. Accelerated EGO obtains comparable cross validation accuracy with other methods and the CPU time can be reduced a lot due to the parallel computation and sampling method. [ABSTRACT FROM AUTHOR]

In this work, we predict the outcomes of high fidelity multivariate computer simulations from low fidelity counterparts using function-to-function regression. The high fidelity simulation takes place on a high definition mesh, while its low fidelity counterpart takes place on a coarsened and truncated mesh. We showcase our approach by applying it to a complex finite element simulation of an abdominal aortic aneurysm which provides the displacement field of a blood vessel under pressure. In order to link the two multidimensional outcomes we compress them and then fit a function-to-function regression model. The data are high dimensional but of low sample size, meaning that only a few simulations are available, while the output of both low and high fidelity simulations is in the order of several thousands. To match this specific condition our compression method assumes a Gaussian Markov random field that takes the finite element geometry into account and only needs little data. In order to solve the function-to-function regression model we construct an appropriate prior with a shrinkage parameter which follows naturally from a Bayesian view of the Karhunen–Loève decomposition. Our model enables real multivariate predictions on the complete grid instead of resorting to the outcome of specific points. [ABSTRACT FROM AUTHOR]

Tuning a complex simulation code refers to the process of improving the agreement of a code calculation with respect to a set of experimental data by adjusting parameters implemented in the code. This process belongs to the class of inverse problems or model calibration. For this problem, the approximated nonlinear least squares (ANLS) method based on a Gaussian process (GP) metamodel has been employed by some researchers. A potential drawback of the ANLS method is that the metamodel is built only once and not updated thereafter. To address this difficulty, we propose an iterative algorithm in this study. In the proposed algorithm, the parameters of the simulation code and GP metamodel are alternatively re-estimated and updated by maximum likelihood estimation and the ANLS method. This algorithm uses both computer and experimental data repeatedly until convergence. A study using toy-models including inexact computer code with bias terms reveals that the proposed algorithm performs better than the ANLS method and the conditional-likelihood-based approach. Finally, an application to a nuclear fusion simulation code is illustrated. [ABSTRACT FROM AUTHOR]

The problem of integrating multifidelity data has been studied extensively, due to integrated analyses being able to provide better results than separately analyzing various data types. One popular approach is to use linear autoregressive models with location- and scale-adjustment parameters. Such parameters are typically modeled using stationary Gaussian processes. However, the stationarity assumption may not be appropriate in real-world applications. To introduce nonstationarity for enhanced flexibility, we propose a novel integration model based on deep Gaussian processes that can capture nonstationarity via successive warping of latent variables through multiple layers of Gaussian processes. For inference of the proposed model, we use a doubly stochastic variational inference algorithm. We validate the proposed model using simulated and real-data examples. [ABSTRACT FROM AUTHOR]

In reliability engineering studies, computer codes are increasingly used to model physical phenomena which, in many cases, can be very time-consuming to run. A widely accepted approach consists in approximating the CPU-time expensive computer model by a surrogate model. One of the most popular surrogate model is the Gaussian Process regression, as it provides, additionally to a prediction at an unobserved point, an uncertainty around this prediction (a predictive distribution). However, in practice, the quality of this metamodel depends on several choices, as the estimation and validation algorithms. The present work aims at proposing a new algorithm, based on constrained optimization multi-objective techniques, to estimate the Gaussian process hyperparameters in order to ensure robust and accurate (i.e. reliable) predictive distribution of the Gaussian process. An intensive numerical benchmark on various analytical functions, with different input dimensions and learning sample sizes, shows its good performance in comparison with standard estimation algorithms. The new algorithm is also applied to a real test case modeling an aquatic ecosystem. It is compared with a recent robust and sophisticated Bayesian method; it proves to be as efficient while being less sensitive to the specification of the Gaussian process model. • New algorithm for obtaining robust estimation of the Gaussian process metamodel hyperparameters. • It jointly maximizes the likelihood and empirical coverage function of GP prediction intervals. • It is validated on a several analytical test functions of variable dimension (1 to 20). • The application relevance of this algorithm is shown on a real aquatic ecosystem model. • It competes with Bayesian methods while being less sensitive to certain tuning choices. [ABSTRACT FROM AUTHOR]

In the framework of risk assessment, computer codes are increasingly used to understand, model and predict physical phenomena. As these codes can be very time-consuming to run, which severely limit the number of possible simulations, a widely accepted approach consists in approximating the CPU-time expensive computer model by a so-called "surrogate model". In this context, the Gaussian Process regression is one of the most popular technique. It offers the advantage of providing a predictive distribution for all new evaluation points. An uncertainty associated with any quantity of interest (e.g. a probability of failure in reliability studies) to be estimated can thus be deduced and adaptive strategies for choosing new points to run with respect to this quantity can be developed. This paper focuses on the estimation of the Gaussian process covariance parameters by reviewing recent works on the analysis of the advantages and disadvantages of usual estimation methods, the most relevant validation criteria (for detecting poor estimation) and recent robust and corrective methods. • Different estimation methods of hyperparameters of the Gaussian process metamodel are reviewed. • Most of estimation methods lead to good predictivity, but with poor quality prediction intervals. • Several adequate metrics are described for Gaussian process predictive distribution validation. • Bayesian estimation approaches are theoretically very attractive, but may be intractable. • Approaches relying on ad-hoc corrections to have reliable prediction intervals may be irrelevant. [ABSTRACT FROM AUTHOR]

This article deals with the sequential design of experiments for (deterministic or stochastic) multi-fidelity numerical simulators, that is, simulators that offer control over the accuracy of simulation of the physical phenomenon or system under study. Accurate simulations usually entail a high computational effort, while coarse simulations are obtained at a lower cost. The cost can be measured, for example, by the run time of the simulator or the financial cost of the computing resources. In this setting, simulation results obtained at several levels of fidelity can be combined in order to estimate quantities of interest (the optimal value of the output, the probability that the output exceeds a given threshold, etc.) in an efficient manner. We propose a new Bayesian sequential strategy called maximal rate of stepwise uncertainty reduction (MR-SUR), that selects additional simulations to be performed by maximizing the ratio between the expected reduction of uncertainty and the cost of simulation. This generic strategy unifies several existing methods, and provides a principled approach to develop new ones. We assess its performance on several examples, including a computationally intensive problem of fire safety analysis where the quantity of interest is the probability of exceeding a tenability threshold during a building fire. [ABSTRACT FROM AUTHOR]

Multi‐fidelity simulations are widely employed in engineering. When the simulators are time consuming to run, an autoregressive Gaussian process (AGP) model fitted with data from a nested space‐filling design can be employed as emulator. However, the AGP model assumes the simulators at different levels of fidelity share the same inputs. This article considers bi‐fidelity simulations with a high‐fidelity (HF) simulator and a low‐fidelity (LF) simulator, where the HF simulator contains a vector of inputs not shared with the LF simulator, called augmented input. The augmented input captures finer modeling details neglected by the LF simulator, and the HF simulator reduces to the LF simulator when some or all components of the augmented input tend to zero. To ensure this boundary constraint in the domain of the augmented input is satisfied, we propose a modified AGP model that uses covariance functions (CFs) constructed from covariances of integrated stochastic processes, called integrated CFs. Five families of integrated CFs are compared in two numerical examples based on finite element simulators and in numerical simulations based on four test functions with analytical forms. It is demonstrated that certain choices of integrated CFs yield substantial improvements in prediction performance attained by the modified AGP model. Matlab codes for reproducing reported results are given in the Supporting Information. [ABSTRACT FROM AUTHOR]

Computer experiments are widely used to mimic expensive physical processes as black-box functions. A typical challenge of expensive computer experiments is to find the set of inputs that produce the desired response. This study proposes a multi-armed bandit regularized expected improvement (BREI) method to adaptively adjust the balance between exploration and exploitation for efficient global optimization of long-running computer experiments with low noise. The BREI adds a stochastic regularization term to the objective function of the expected improvement to integrate the information of additional exploration and exploitation into the optimization process. The proposed study also develops a multi-armed bandit strategy based on Thompson sampling for adaptive optimization of the tuning parameter of the BREI based on the preexisting and newly tested points. The performance of the proposed method is validated against some of the existing methods in the literature under different levels of noise using a case study on optimization of the collision avoidance algorithm in mobile robot motion planning as well as extensive simulation studies.

Robustness analysis is an emerging field in the uncertainty quantification domain. It involves analyzing the response of a computer model—which has inputs whose exact values are unknown—to the perturbation of one or several of its input distributions. Practical robustness analysis methods therefore require a coherent methodology for perturbing distributions; we present here one such rigorous method, based on the Fisher distance on manifolds of probability distributions. Further, we provide a numerical method to calculate perturbed densities in practice which comes from Lagrangian mechanics and involves solving a system of ordinary differential equations. The method introduced for perturbations is then used to compute quantile-related robustness indices. We illustrate these "perturbed-law based" indices on several numerical models. We also apply our methods to an industrial setting: the simulation of a loss of coolant accident in a nuclear reactor, where several dozen of the model's physical parameters are not known exactly, and where limited knowledge on their distributions is available. [ABSTRACT FROM AUTHOR]