Search

Discrete-event simulation models generate random variates from input distributions and compute outputs according to the simulation logic. The input distributions are typically fitted to finite real-world data and thus are subject to estimation errors that can propagate to the simulation outputs: an issue commonly known as input uncertainty (IU). This paper investigates quantifying IU using the output confidence intervals (CIs) computed from bootstrap quantile estimators. The standard direct bootstrap method has overcoverage due to convolution of the simulation error and IU; however, the brute-force way of washing away the former is computationally demanding. We present two new bootstrap methods to enhance direct resampling in both statistical and computational efficiencies using shrinkage strategies to down-scale the variabilities encapsulated in the CIs. Our asymptotic analysis shows how both approaches produce tight CIs accounting for IU under limited input data and simulation effort along with the simulation sample-size requirements relative to the input data size. We demonstrate performances of the shrinkage strategies with several numerical experiments and investigate the conditions under which each method performs well. We also show advantages of nonparametric approaches over parametric bootstrap when the distribution family is misspecified and over metamodel approaches when the dimension of the distribution parameters is high. History: Accepted by Bruno Tuffin, Area Editor for Simulation. Funding: This work was supported by the National Science Foundation [CAREER CMMI-1834710, CAREER CMMI-2045400, DMS-1854659, and IIS-1849280]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information (https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0044) as well as from the IJOC GitHub software repository (https://github.com/INFORMSJoC/2022.0044). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/. [ABSTRACT FROM AUTHOR]

We consider the problem of deriving confidence intervals for the mean response of a system that is represented by a stochastic simulation whose parametric input models have been estimated from [...]

Perhaps the most frequently asked questions we receive have to do with where does a particular proposal fit, i.e., where should it be submitted in NSF. Among the three programs [...]

In this paper, we provide a review of statistical methods that are useful in conducting computer experiments. Our focus is on the task of metamodeling, which is driven by the goal of optimizing a complex system via a deterministic simulation model. However, we also mention the case of a stochastic simulation, and examples of both cases are discussed. The organization of our review first presents several engineering applications, it then describes approaches for the two primary tasks of metamodeling: (i) selecting an experimental design; and (ii) fitting a statistical model. Seven statistical modeling methods are included. Both classical and newer experimental designs are discussed. Finally, our own computational study tests the various metamodeling options on two two-dimensional response surfaces and one ten-dimensional surface., 1. Introduction In the design of complex systems, computer experiments are frequently the only practical approach to obtaining a solution. Typically, a simulation model of system performance is constructed based [...]

In many scientific and engineering domains, it is common to analyze and simulate complex physical systems using mathematical models. Although computing resources continue to increase in power and speed, computer simulation and analysis codes continue to grow in complexity and remain computationally expensive, limiting their use in design and optimization. Consequently, many researchers have developed different metamodeling strategies to create inexpensive approximations of computationally expensive computer simulations. These approximations introduce a new element of uncertainty during design optimization, and there is a need to develop efficient methods to assess metamodel validity. We investigate computationally inexpensive assessment methods for metamodel validation based on leave-k-out cross validation and develop guidelines for selecting k for different types of metamodels. Based on the results from two sets of test problems, k = 1 is recommended for leave-k-out cross validation of low-order polynomial and radial basis function metamodels, whereas k = 0.1N or [check] N is recommended for kriging metamodels, where N is the number of sample points used to construct the metamodel.

A new approach, based on global response surface (GRS) modeling techniques, is used to design two-layer electro-magnetically coupled rectangular patch antennas. Data from a grid of design points is used to construct a model of the input impedance of the antenna. The model is then optimized to provide the best design geometries for the frequency band of interest. Three examples are presented to validate the method. Index Terms - Microstrip antennas.

A reliability-testing information system relational database for an accelerated life-testing facility and its command structure design are described. The database design, which generated consistent reliability test results, enhanced data integrity. The design provided information on the reporting process and reliability-data collection automation.