Search

International audience; Tasks such as analysis, design optimization, and uncertainty quantification can be computationally expensive. Surrogate modeling is often the tool of choice for reducing the burden associated with such data-intensive tasks. However, even after years of intensive research, surrogate modeling still involves a struggle to achieve maximum accuracy within limited resources. This work summarizes various advanced, yet often straightforward, statistical tools that help. We focus on four techniques with increasing popularity in the surrogate modeling community: (i) variable screening and dimensionality reduction in both the input and the output spaces, (ii) data sampling techniques or design of experiments, (iii) simultaneous use of multiple surrogates, and (iv) sequential sampling. We close the paper with some suggestions for future research.

Computer simulations are used to model of complex physical systems. Often, these models represent the solutions (or at least approximations) to partial differential equations that are obtained through costly numerical integration. This paper presents a survey of two important statistical/machine learning approaches that have shaped the field of scientific modeling. Firstly we survey the developments on Bayesian calibration of computer models since the seminal work by Kennedy and O’Hagan. In their paper, the authors proposed an elegant way to use the Gaussian processes to extend calibration beyond parameter and observation uncertainty and include model-form and data size uncertainty. Secondly, we also survey physics-informed neural networks, a topic that has been receiving growing attention due to the potential reduction in computational cost and modeling flexibility. In addition, in order to help the interested reader to familiarize with these topics and venture into custom implementations, we present a summary of applications and software tools. Finally, we close the paper with suggestion for future research directions and a thought provoking call for action.

Maintenance of engineering assets (for example, aircraft, jet engines, and wind turbines) is a profitable business. Unfortunately, building models that estimate remaining useful life for large flee...

Lithium-ion batteries are commonly used to power electric unmanned aircraft vehicles (UAVs).Therefore, the ability to model both the state of charge as well as battery health is very important for reliable and affordable operation of UAV fleets.Even though models based on first principles are accurate and trustworthy, the complex electro-chemistry that governs battery discharge and aging makes it hard to build and use such models for in-time monitoring of battery conditions.Moreover, the careful tuning or estimation of high-fidelity model parameters hampers the straightforward deployment in the field.Alternatively, reduced order models have the advantage of capturing the overall behavior of battery discharge. Reduced-order principle-based models are built by carefully simplifying the physics/chemistry such that computational cost is dramatically reduced while the overall behavior of the system is still captured.These simplifications also lead to a number of parameters to be estimated based on data as well as residual discrepancy (model-form uncertainty).This approach can lead to a number of parameters to be estimated based on data as well as residual model-form uncertainty; a property shared with machine learning models. The latter are solely built on the basis of data, and can still capture unexpected nonlinearities.The drawback is that traditional machine learning tends to require large number of data points hard to retrieve in many scientific and engineering fields like, for example, the field of battery discharge and degradation prediction. In this paper, we will present a hybrid modeling approach for tracking and forecasting battery aging based on ``as-used'' conditions.Our approach directly implements a reduced-order model based on Nerst and Butler-Volmer equations within a deep neural network framework.While most of the input-output relationship is captured by reduced-order models, the data-driven kernels reduce the gap between predictions and observations.The hybrid model estimates the overall battery discharge, and a multilayer perceptron models the battery internal voltage.Battery aging is characterized by time-dependent internal resistance and the amount of available Li-ions.We address the difficult issue of building and updating the aging model by reducing the need for reference discharge cycles.This is beneficial to operators, since it reduces the need of taking the batteries out of commission.We compensate for lack of reference discharge cycles by using a probabilistic model that leverages previously available fleet-wide information. We validate our approach using data publicly available through the NASA Prognostics Center of Excellence website.Results showed that our hybrid battery prognosis model can be successfully calibrated, even with a limited number of observations.Moreover, the model can help optimizing battery operation by offering long-term forecast of battery capacity.

The operation of complex systems can drift away from the initial design conditions, due to environmental conditions, equipment wear or specific restrictions. Steam generators are complex equipment and their proper operation relies on the identification of their most relevant parameters. An approach to rank the operational parameters of a subcritical steam generator of an actual 360 MW power plant is presented. An Artificial Neural Network - ANN delivers a model to estimate the steam generator efficiency, electric power generation and flue gas outlet temperature as a function of seven input parameters. The ANN is trained with a two-year long database, with training errors of 0.2015 and 0.2741 (mean absolute and square error) and validation errors of 0.32% and 2.350 (mean percent and square error). That ANN model is explored by means of a combination of situations proposed by a Design of Experiment - DoE approach. All seven controlled parameters showed to be relevant to express both steam generator efficiency and electric power generation, while primary air flow rate and speed of the dynamic classifier can be neglected to calculate flue gas temperature as they are not statistically significant. DoE also shows the prominence of the primary air pressure in respect to the steam generator efficiency, electric power generation and the coal mass flow rate for the calculation of the flue gas outlet temperature. The ANN and DoE combined methodology shows to be promising to enhance complex system efficiency and helpful whenever a biased behavior must be brought back to stable operation.

The Coffin–Manson–Basquin–Haford (CMBH) model is a well-accepted strain-life relationship to model fatigue life as a function of applied strain. In this paper, we propose a nonstationary uncertainty model for the CMBH model, alongside a Bayesian framework for model calibration and estimation of confidence and prediction intervals. Using Inconel 617 coupon test data, we compared our approach to traditional stationary variance models. The proposed uncertainty model successfully captures the fact that the variance of strain amplitude decreases with increasing fatigue life. Additionally, a discussion on how to use the proposed Bayesian framework to compensate for the lack of data by using prior information coming from similar alloys is also presented considering Hastealloy-X and Inconel 617 coupon data.

We present a physics-informed neural network modeling approach for missing physics estimation in cumulative damage models. This hybrid approach is designed to merge physics-informed and data-driven layers within deep neural networks. The result is a cumulative damage model in which physics-informed layers are used to model relatively well understood phenomena and data-driven layers account for hard-to-model physics. A numerical experiment is used to present the main features of the proposed framework. The test problem consists of predicting corrosion-fatigue of an Al 2024-T3 alloy used on panels of aircraft wings. Besides cyclic loading, panels are also subjected to saline corrosion. In this case, physics-informed layers implement the well-known Walker model for crack propagation, while data-driven layers are trained to compensate the bias in damage accumulation due to the corrosion effects. The physics-informed neural network is trained using full observation of inputs (far-field loads, stress ratio, and a corrosivity index defined per airport) and very limited observation of outputs (crack length at inspection for only a small portion of the fleet). Results show that the physics-informed neural network is able to learn how to compensate the missing physics of corrosion in the original fatigue model. Predictions from the hybrid model can be used in fleet management, for example, to prioritize inspection across the fleet or forecast ahead of time the number of planes with damage above a threshold.

Lithium-ion batteries are commonly used to power unmanned aircraft vehicles (UAVs). The ability to model and forecast remaining useful life of these batteries enables UAV reliability assurance. Building principled accurate models is challenging due to the complex electrochemistry that governs battery operation. Alternatively, reduced order models have the advantage of capturing the overall behavior of battery discharge, although they suffer from simplifications and residual discrepancy. This paper presents a hybrid modeling approach that directly implements physics within deep neural networks. While most of the input–output relationship is captured by reduced-order models, data-driven kernels reduce the gap between predictions and observations. A reduced-order model based on Nernst and Butler–Volmer equations represents the overall battery discharge, and a multilayer perceptron models the battery non-ideal voltage. Battery aging is characterized by time-dependent internal resistance and the amount of available Li-ions, which are modeled through an ensemble of variational Bayesian multilayer perceptrons. The approach is validated using data publicly available through the NASA Prognostics Center of Excellence website. Results showed that our hybrid battery prognosis model can be successfully calibrated, even with a limited number of observations. Moreover, the model can help optimizing battery operation by offering long-term forecast of battery capacity.

Prognosis and health management plays an important role in the control of costs associated with operating large industrial equipment, such as wind turbines and aircraft. It is only fair that engineers and scientists have vastly researched modeling approaches to support decision making. Motivated by the growing availability of data and computational power as well as the advances in algorithms and methods, modeling frameworks often merge elements of physics, machine learning, and statistical learning. In this paper, we present a review on modeling in support of prognosis and health management of industrial equipment. This survey complements the existing prognosis and health management literature by discussing how modeling strategies are influenced by industry-specific aspects such as maintenance approaches (e.g., reactive, proactive, and predictive), implementation factors (e.g., industry, business model, purpose, development, and deployment), as well as supporting technologies (sensing, repair, and modeling itself). We use the onshore wind energy and civil aviation industries to illustrate how these aforementioned aspects can influence modeling and implementation of prognosis and health management. The literature review is broad and covers contributions over the past 40 years. We close the paper with few topics that can motive research going forward.

In this paper, we present a novel physics-informed neural network modeling approach for corrosion-fatigue. The hybrid approach is designed to merge physics- informed and data-driven layers within deep neural networks. The result is a cumulative damage model where the physics-informed layers are used to model the relatively well understood physics (crack growth through Paris law) and the data-driven layers account for the hard to model effects (bias in damage accumulation due to corrosion). A numerical experiment is used to present the main features of the proposed physics-informed recurrent neural network for damage accumulation. The test problem consists of predicting corrosion-fatigue of an Al 2024-T3 alloy used on panels of aircraft wing. Besides cyclic loading, the panels are also subjected to saline corrosion. The physics-informed neural network is trained using full observation of inputs (far-field loads, stress ratio and a corrosivity index – defined per airport) and very limited observation of outputs (crack length at inspection for only a small portion of the fleet). Results show that the physics-informed neural network is able to learn the correction in the original fatigue model due to corrosion and predictions are accurate enough for ranking damage in different airplanes in the fleet (which can be used to prioritizing inspection).

Unexpected main bearing failure on a wind turbine causes unwanted maintenance and increased operation costs (mainly due to crane, parts, labor, and production loss). Unfortunately, historical data indicates that failure can happen far earlier than the design lives of the component. Root cause analysis investigations have pointed to problems inherent from manufacturing as the major contributor, as well as issues related to event loads (e.g., startups, shutdowns, and emergency stops), extreme environmental conditions, and maintenance practices, among others. Altogether, the multiple failure modes and contributors make modeling remaining useful life of main bearings a very daunting task. In this paper, we present a novel physics-informed neural network modeling approach for main bearing fatigue. The proposed approach is fully hybrid and designed to merge physics- informed and data-driven layers within deep neural networks. The result is a cumulative damage model where the physics-informed layers are used model the relatively well-understood physics (L10 fatigue life) and the data-driven layers account for the hard to model components (e.g., contribution due to poor greasing conditions).

The practice of applying strain-life relationships to model materials subjected to low cycle fatigue conditions has been widely-accepted for the past seventy years. The Coffin-Manson rule employs a double, two-parameter power law equation to correlate cycles to failure to strain range (or vice versa). Plastic strain range dominates low life, while elastic strain dominates high life. In some settings with well-established materials, copious amounts of test data are available. The median response and the Coffin-Manson parameters are determined through regression. Scatter in both strain amplitude and fatigue life values are a consequence of material and specimen variation, test technician attributes or more. Recent initiatives have endeavored to transition deterministic approaches over to probabilistic analogies. Confidence bands, lower ones in particular, are useful for understanding reliability curves and implementing reliability-based design and optimization. In settings with accelerated product development schedules, intentionally sparse sets of test data are used to play “what if” scenarios in the context of life prediction. We propose a non-stationary variance to model deviations from the Coffin-Manson rule and use Bayesian statistical methods to estimate model parameter uncertainties. The proposed approach is applied to the calibration of strain-life parameters of the candidate material Inconel 617, a Ni-base alloy used is combustion equipment. When compared with constant variance (such as in traditional regression), the results show an improved characterization of the confidence bounds for fatigue life. This is important as it indicates that the methodology can be used to manage the number of coupon test to achieve an acceptable convergence.

Services and warranties of large fleets of engineering assets is a very profitable business. The success of companies in that area is often related to predictive maintenance driven by advanced analytics. Therefore, accurate modeling, as a way to understand how the complex interactions between operating conditions and component capability define useful life, is key for services profitability. Unfortunately, building prognosis models for large fleets is a daunting task as factors such as duty cycle variation, harsh environments, inadequate maintenance, and problems with mass production can lead to large discrepancies between designed and observed useful lives. This paper introduces a novel physics-informed neural network approach to prognosis by extending recurrent neural networks to cumulative damage models. We propose a new recurrent neural network cell designed to merge physics-informed and data-driven layers. With that, engineers and scientists have the chance to use physics-informed layers to model parts that are well understood (e.g., fatigue crack growth) and use data-driven layers to model parts that are poorly characterized (e.g., internal loads). A simple numerical experiment is used to present the main features of the proposed physics-informed recurrent neural network for damage accumulation. The test problem consist of predicting fatigue crack length for a synthetic fleet of airplanes subject to different mission mixes. The model is trained using full observation inputs (far-field loads) and very limited observation of outputs (crack length at inspection for only a portion of the fleet). The results demonstrate that our proposed hybrid physics-informed recurrent neural network is able to accurately model fatigue crack growth even when the observed distribution of crack length does not match with the (unobservable) fleet distribution., Data and codes (including our implementation for both the multi-layer perceptron, the stress intensity and Paris law layers, the cumulative damage cell, as well as python driver scripts) used in this manuscript are publicly available on GitHub at https://github.com/PML-UCF/pinn. The data and code are released under the MIT License

A number of physical systems can be described by ordinary differential equations. When physics is well understood, the time dependent responses are easily obtained numerically. The particular numerical method used for integration depends on the application. Unfortunately, when physics is not fully understood, the discrepancies between predictions and observed responses can be large and unacceptable. In this paper, we propose an approach that uses observed data to estimate the missing physics in the original model (i.e., model-form uncertainty). In our approach, we first design recurrent neural networks to perform numerical integration of the ordinary differential equations. Then, we implement the recurrent neural network as a directed graph. This way, the nodes in the graph represent the physics-informed kernels found in the ordinary differential equations. We quantify the missing physics by carefully introducing data-driven in the directed graph. This allows us to estimate the missing physics (discrepancy term) even for hidden nodes of the graph. We studied the performance of our proposed approach with the aid of three case studies (fatigue crack growth, corrosion-fatigue crack growth, and bearing fatigue) and state-of-the-art machine learning software packages. Our results demonstrate the ability to perform estimation of discrepancy, reducing gap between predictions and observations, at reasonable computational cost.

Field failures of wind turbine main bearings yield to undesired downtime and significant maintenance costs. Fatigue failure is a dominant mode for legacy turbines, which can be expressed with physics-informed models to some extent. However, these models often inherent large uncertainties due to unknown lubricant degradation mechanism. Therefore, periodical assessment of the grease state plays a crucial role in calibration of bearing fatigue models. As opposed to detailed laboratory analysis, grease visual inspection can lead to large uncertainties in characterization of grease condition (although visual inspection can be cost and time effective). In this paper, we introduce a hybrid model for main bearing fatigue damage accumulation and calibrated using only visual grease inspections. In our hybrid model, bearing fatigue damage portion consists of known physics formulations, and unknown grease degradation is represented with deep neural networks. In addition, we introduce a custom tailored classifier that enables the model to map from damage scale to visual rankings. Results showed that the bearing fatigue prognosis model can be successfully calibrated, even with limited and noisy field observations. Moreover, the model can help optimizing park reliability by suggesting turbine-specific regreasing intervals. The source codes and links to the data can be found in the following GitHub repository https://github.com/PML-UCF/pinn_wind_bearing .

In the service market targeting fleets of industrial assets (e.g., aircraft, jet engines, wind turbines, etc.), original equipment manufacturers and service providers compete with one another through offers covering day-to-day service as well as major maintenance and repairs over. Since decision-making is highly guided by reliability models, it is safe to say that services profitability dependents on the ability to understand the complex stochastic interactions between operating conditions and component capability. Unfortunately, factors such as aggressive mission mixes introduced by operators, exposure to a harsh environment, inadequate maintenance, and problems with mass production can lead to large discrepancies between predicted and observed useful lives. This paper focuses on the quantification of the infant mortality impact on fleets of industrial assets. A numerical experiment is used to study how the number of failure observations and fleet size impacts the modeling of fleet reliability. Dynamic Bayesian networks implementing physics-based models are used to model fleet unreliability considering the effects of bad batch of materials.The results demonstrate that material capability, penetration of bad batch of material in the fleet, and fleet size drastically influence the model accuracy. Therefore, small fleet operators, which naturally observe a lownumber of failures, have to deal with larger uncertainties when quantifying infant mortality. This negatively impacts their ability to allocate resources such as inventory, labor, and account for the loss of productivity while servicing their fleet. With large fleet operators, on the other hand, large number of failure observations can cause high financial burden. Nevertheless, it also allows for reduced uncertainty in building/updating the reliability models, which can help their ability to forecast future failures and make provisions for service and maintenance. Finally, the results also show that measures such as recommissioning of the fleet and inspection campaigns can mitigate the effects of fleet-wide early life problems.

Available historical field data shows that wind turbine main bearing failure can lead to major operation and maintenance costs due to unscheduled downtime. For legacy turbines, fa- tigue is one of the major failure modes and, to a degree, can be partially modeled with physics-based formulations. Unfor- tunately, existing bearing fatigue models can potentially be inaccurate due to lack of understanding of the lubricant degra- dation. One way to enhance these models is to track the grease damage along with the bearing fatigue damage. However, the need of grease degradation data can become an impedi- ment for such strategy. In this paper, we will demonstrate that it is possible to calibrate grease degradation models with cost-efficient periodic visual inspections. Knowing that such inspections introduce observation uncertainty to the model, we will use a hybrid physics-informed deep neural networks to quantify such uncertainties within our models. We built a hybrid model that fuses the physics-based understanding of the bearing fatigue failure with the ability of data-driven layers to compensate the missing physics, with respect to the grease degradation. The proposed hybrid model is also ca- pable of decoding uncertain visual grease inspections with a custom designed classifier. We illustrate the merits of the model with the support of case studies, where we test inspec- tion with different levels of conservatism to train the model and compare the predictions of these models on an artificial wind park. Results from the case studies indicate the success- ful prognostic performance of the trained with limited and noisy observations. While grease damage is predicted with 0.3% root mean square error as a result of baseline inspection campaign, bearing life is prediction is conservatively off only by months for aggressive turbines that have 10 years of life.

We present a tutorial on how to directly implement integration of ordinary differential equations through recurrent neural networks using Python. In order to simplify the implementation, we leveraged modern machine learning frameworks such as TensorFlow and Keras. Besides, offering implementation of basic models (such as multilayer perceptrons and recurrent neural networks) and optimization methods, these frameworks offer powerful automatic differentiation. With all that, the main advantage of our approach is that one can implement hybrid models combining physics-informed and data-driven kernels, where data-driven kernels are used to reduce the gap between predictions and observations. Alternatively, we can also perform model parameter identification. In order to illustrate our approach, we used two case studies. The first one consisted of performing fatigue crack growth integration through Euler’s forward method using a hybrid model combining a data-driven stress intensity range model with a physics-based crack length increment model. The second case study consisted of performing model parameter identification of a dynamic two-degree-of-freedom system through Runge–Kutta integration. The examples presented here as well as source codes are all open-source under the GitHub repository https://github.com/PML-UCF/pinn_code_tutorial .

Services and warranties of large fleets of engineering assets is a very profitable business where original equipment manufacturers and independent service providers offer contracts designed to cover events in day-to-day service as well as major maintenance and repairs over the life of the asset. Accurate reliability modeling, as a way to understand how the complex stochastic interactions between operating conditions and component capability define useful life, is key for services profitability. The modeling task is daunting as factors such as aggressive mission mixes introduced by operators, exposure to harsh environment, inadequate maintenance, and problems with mass production (bad batch of materials) can lead to large discrepancies between designed and observed useful lives. This paper is focused on how to quantify the impact of infant mortality in fleets of industrial assets. A simple numerical experiment is used to address the fundamental question: how does number of observations and fleet size interact with each other in fleet management? The results demonstrate that material capability, penetration of bad batch of material in the fleet, and commissioning time can drastically influence fleet unreliability. Moreover, infant mortality due to manufacturing problems/material capability is a manifestation of an outlier problem. As a consequence, the propensity to observe first failures depend on the actual fleet size. Since failure observations are used to build/update the reliability models, small fleet operators have to deal with large uncertainties when quantifying infant mortality. This impacts their ability to make provisions for service and maintenance (inventory, labor, loss of productivity, etc.). Although the large number of failure observations causes a financial burden in large fleet operators, it also allows for reduced uncertainty in building/updating the reliability models. In turn, this improves their ability to forecast future failures and make provisions for service and maintenance.

The growing power of computers enabled techniques created for design and analysis of simulations to be applied to a large spectrum of problems and to reach high level of acceptance among practitioners. Generally, when simulations are time consuming, a surrogate model replaces the computer code in further studies (e.g., optimization, sensitivity analysis, etc.). The first step for a successful surrogate modeling and statistical analysis is the planning of the input configuration that is used to exercise the simulation code. Among the strategies devised for computer experiments, Latin hypercube designs have become particularly popular. This paper provides a tutorial on Latin hypercube design of experiments, highlighting potential reasons of its widespread use. The discussion starts with the early developments in optimization of the point selection and goes all the way to the pitfalls of the indiscriminate use of Latin hypercube designs. Final thoughts are given on opportunities for future research. Copyright © 2015 John Wiley & Sons, Ltd.

The use of metamodeling techniques in the design and analysis of computer experiments has progressed remarkably in the past 25 years, but how far has the field really come? This is the question addressed in this paper, namely, the extent to which the use of metamodeling techniques in multidisciplinary design optimization have evolved in the 25 years since the seminal paper on design and analysis of computer experiments by Sacks et al. (“Design and Analysis of Computer Experiments,” Statistical Science, Vol. 4, No. 4, 1989, pp. 409–435). Rather than a technical review of the entire body of metamodeling literature, the focus is on the evolution and motivation for advancements in metamodeling with some discussion on the research itself; not surprisingly, much of the current research motivation is the same as it was in the past. Based on current research thrusts in the field, multifidelity approximations and ensembles (i.e., sets) of metamodels, as well as the availability of metamodels within commercial soft...

Lightweight design of vehicle structures parameters under crashworthiness is hard to accomplish because of the complexity of simulations required in crash analysis. To reduce the computation demand, surrogates (metamodels) are often used in place of the actual simulation models in design optimization to fit the mathematical relationship between design variables and responses. Each optimization cycle consists of analyzing a number of designs, fitting surrogates for the responses, performing optimization based on the surrogates for a candidate optimum, and finally analyzing that candidate. Even so, optimization using crash analysis codes is often allowed to run only for very few cycles. While traditional surrogate is unbiased which means prediction values at half region is lower than actual values, predicted candidate optimum usually is not feasible after validating by crash simulation. This paper explores the use of conservative surrogates for safe estimations of crashworthiness responses (e.g., intrusion and peak acceleration). We use safety margins to conservatively compensate for fitting errors associated with surrogates. Conservative surrogates minimize the risks associated with underestimation of the responses, which helps push optimization toward the feasible region of the design. We also propose an approach for sequential relaxation of the safety margins allowing for further weight minimization. The approach was tested on the lightweight design of a vehicle subjected to the full-overlap frontal crash. We compare this approach with the traditional use of unbiased surrogates (that is, without adding any safety margin). We find that conservative surrogates successfully drive optimization toward the feasible region of a design space, while that is not always the case with unbiased surrogates.

Modeling fatigue crack growth is computationally challenging because the crack growth rate can only be evaluated at the current crack size. Therefore, the forward Euler method has been a common choice in integrating fatigue crack growth. However, since its accuracy can only be guaranteed with a small step size, the method cannot be applied to the investigation of systems with complex geometry (calling for expensive finite element simulations). Higher-order integration methods, such as the midpoint method, allow larger step size but require evaluation of crack growth rate at crack sizes larger than the current one. In arbitrary geometry, this is not an easy task because the direction of crack growth is unknown in advance. In this paper, surrogate models are generated for the prior crack growth direction and stress intensity factor data. These surrogates are cheap to evaluate and predict the crack growth rate without the need of additional finite element simulations. The step size for the numerical integration is chosen based on the accuracy of the extrapolated crack growth predictions for direction and stress intensity factor. Several examples were tested in which crack growth follows linear and curved paths under a range of boundary conditions leading to different relationships between stress intensity factor and crack size. Results showed that a large increase in the allowable step size may be used with increased accuracy over the Euler method with the need for fewer expensive function evaluations.

Surrogate-based optimization proceeds in cycles. Each cycle consists of analyzing a number of designs, fitting a surrogate, performing optimization based on the surrogate, and finally analyzing a candidate solution. Algorithms that use the surrogate uncertainty estimator to guide the selection of the next sampling candidate are readily available, e.g., the efficient global optimization (EGO) algorithm. However, adding one single point at a time may not be efficient when the main concern is wall-clock time (rather than number of simulations) and simulations can run in parallel. Also, the need for uncertainty estimates limits EGO-like strategies to surrogates normally implemented with such estimates (e.g., kriging and polynomial response surface). We propose the multiple surrogate efficient global optimization (MSEGO) algorithm, which adds several points per optimization cycle with the help of multiple surrogates. We import uncertainty estimates from one surrogate to another to allow use of surrogates that do not provide them. The approach is tested on three analytic examples for nine basic surrogates including kriging, radial basis neural networks, linear Shepard, and six different instances of support vector regression. We found that MSEGO works well even with imported uncertainty estimates, delivering better results in a fraction of the optimization cycles needed by EGO.

We demonstrate the use of multiple surrogates and kriging believer for parallelizing surrogate-based contour estimation. For the demonstration example, we reduce wall clock time with minimal penalty in number of simulations.

The dielectric barrier discharge (DBD) plasma actuator offers promising opportunities for flow control because of its fast response and non-moving parts. In this work, surrogate modelling is adopted to better understand the impact of the materials and operational parameters on the actuator performance, and to provide an efficient approach for performance estimation. The DBD model based on 2-species helium chemistry engages three design variables: operating frequency, polarity (positive/negative) time ratio of the applied voltage and the dielectric constant of the insulator. Two objectives are identified: the net force generated and the power requirement. Multiple surrogate models are used, which identify two branches of the Pareto front with opposite net force direction and substantially different parametric sensitivities. Global sensitivity analysis indicates that the voltage frequency and polarity ratio are important in different regions of the design space, while the dielectric constant is always important.

CITATION: Viana, F. A. C., Venter, G. & Balabanov, V. 2010. An algorithm for fast optimal Latin hypercube design of experiments. International Journal for Numerical Methods in Engineering, 82(2):135-156, doi:10.1002/nme.2750.

T HE use of surrogates for facilitating optimization and statistical analysis of computationally expensive simulations has become commonplace [1–4]. They offer easy-to-compute prediction and in some cases (such as kriging [5,6] and polynomial response surfaces [7,8]), they also furnish the prediction variance as a measure of uncertainty [9]. Figure 1a illustrates the concepts of prediction and prediction variance. Adaptive sampling and optimization methods use the prediction variance to select the next sampling point. For example, the Efficient Global Optimization (EGO) [10] and the Enhanced Sequential Optimization [11] algorithms use the kriging prediction variance to seek the point of maximum expected improvement as the next simulation for the optimization process. For such methods, it is important to assess the accuracy of the prediction variance; but presently, this is not available (although there is work on how to improve the uncertainty structure [12]). Cross validation is a standard tool for estimating the mean square errors (see the Appendix), thus the quality of the fit; and it can be used for selecting surrogates in a set [13–15]. Cross validation divides a set of p data points into k subsets. The surrogate isfit to all subsets except one, and the error is checked in the subset that was left out. This process is repeated for all subsets to produce a vector of cross-validation errors, eXV . Figure 1b illustrates cross validation when only one point is omitted. We propose using cross validation for estimating the correlation between the prediction variance and the errors. Specifically we propose to use the correlation between the absolute values of the cross-validation errors and the square root of the prediction variance at the points that were left out. II. Correlation Between Square Root of Prediction Variance and Absolute Errors

Surrogate models are commonly used to replace expensive simulations of engineering problems. Frequently, a single surrogate is chosen based on past experience. This approach has generated a collection of papers comparing the performance of individual surrogates. Previous work has also shown that fitting multiple surrogates and picking one based on cross-validation errors (PRESS in particular) is a good strategy, and that cross-validation errors may also be used to create a weighted surrogate. In this paper, we discussed how PRESS (obtained either from the leave-one-out or from the k-fold strategies) is employed to estimate the RMS error, and whether to use the best PRESS solution or a weighted surrogate when a single surrogate is needed. We also studied the minimization of the integrated square error as a way to compute the weights of the weighted average surrogate. We found that it pays to generate a large set of different surrogates and then use PRESS as a criterion for selection. We found that (1) in general, PRESS is good for filtering out inaccurate surrogates; and (2) with sufficient number of points, PRESS may identify the best surrogate of the set. Hence the use of cross-validation errors for choosing a surrogate and for calculating the weights of weighted surrogates becomes more attractive in high dimensions (when a large number of points is naturally required). However, it appears that the potential gains from using weighted surrogates diminish substantially in high dimensions. We also examined the utility of using all the surrogates for forming the weighted surrogates versus using a subset of the most accurate ones. This decision is shown to depend on the weighting scheme. Finally, we also found that PRESS as obtained through the k-fold strategy successfully estimates the RMSE.

In this work, two optimization methods are investigated to accomplish the parameter identification of the longitudinal motion of a military aircraft within the framework of the Output Error methodology. One of these methods is based on natural algorithms, in particular, the genetic algorithms and particle swarm optimization, which combined constitute the so called Life Cycle Method. The other is the Levenberg-Marquardt optimization algorithm, which is a gradient based method. Since the methodologies differ in the way they perform the optimization, being the first based on search and so appropriate for global minima search and the second gradient based, thus, good for local minima, both are compared and used in such a way that they complement each other.

In this work, a flat pressure bulkhead reinforced by an array of beams is designed using a suite of heuristic optimization methods (Ant Colony Optimization, Genetic Algorithms, Particle Swarm Optimization and LifeCycle Optimization), and the Nelder-Mead simplex direct search method. The compromise between numerical performance and computational cost is addressed, calling for inexpensive, yet accurate analysis procedures. At this point, variable fidelity is proposed as a tradeoff solution. The difference between the low-fidelity and high-fidelity models at several points is used to fit a surrogate that corrects the low-fidelity model at other points. This allows faster linear analyses during the optimization; whilst a reduced set of expensive non-linear analyses are run "off-line," enhancing the linear results according to the physics of the structure. Numerical results report the success of the proposed methodology when applied to aircraft structural components. The main conclusions of the work are (i) the variable fidelity approach enabled the use of intensive computing heuristic optimization techniques; and (ii) this framework succeeded in exploring the design space, providing good initial designs for classical optimization techniques. The final design is obtained when validating the candidate solutions issued from both heuristic and classical optimization. Then, the best design can be chosen by direct comparison of the high-fidelity responses.