Search

International audience; Tasks such as analysis, design optimization, and uncertainty quantification can be computationally expensive. Surrogate modeling is often the tool of choice for reducing the burden associated with such data-intensive tasks. However, even after years of intensive research, surrogate modeling still involves a struggle to achieve maximum accuracy within limited resources. This work summarizes various advanced, yet often straightforward, statistical tools that help. We focus on four techniques with increasing popularity in the surrogate modeling community: (i) variable screening and dimensionality reduction in both the input and the output spaces, (ii) data sampling techniques or design of experiments, (iii) simultaneous use of multiple surrogates, and (iv) sequential sampling. We close the paper with some suggestions for future research.

Computer simulations are used to model of complex physical systems. Often, these models represent the solutions (or at least approximations) to partial differential equations that are obtained through costly numerical integration. This paper presents a survey of two important statistical/machine learning approaches that have shaped the field of scientific modeling. Firstly we survey the developments on Bayesian calibration of computer models since the seminal work by Kennedy and O’Hagan. In their paper, the authors proposed an elegant way to use the Gaussian processes to extend calibration beyond parameter and observation uncertainty and include model-form and data size uncertainty. Secondly, we also survey physics-informed neural networks, a topic that has been receiving growing attention due to the potential reduction in computational cost and modeling flexibility. In addition, in order to help the interested reader to familiarize with these topics and venture into custom implementations, we present a summary of applications and software tools. Finally, we close the paper with suggestion for future research directions and a thought provoking call for action.

Maintenance of engineering assets (for example, aircraft, jet engines, and wind turbines) is a profitable business. Unfortunately, building models that estimate remaining useful life for large flee...

Lithium-ion batteries are commonly used to power electric unmanned aircraft vehicles (UAVs).Therefore, the ability to model both the state of charge as well as battery health is very important for reliable and affordable operation of UAV fleets.Even though models based on first principles are accurate and trustworthy, the complex electro-chemistry that governs battery discharge and aging makes it hard to build and use such models for in-time monitoring of battery conditions.Moreover, the careful tuning or estimation of high-fidelity model parameters hampers the straightforward deployment in the field.Alternatively, reduced order models have the advantage of capturing the overall behavior of battery discharge. Reduced-order principle-based models are built by carefully simplifying the physics/chemistry such that computational cost is dramatically reduced while the overall behavior of the system is still captured.These simplifications also lead to a number of parameters to be estimated based on data as well as residual discrepancy (model-form uncertainty).This approach can lead to a number of parameters to be estimated based on data as well as residual model-form uncertainty; a property shared with machine learning models. The latter are solely built on the basis of data, and can still capture unexpected nonlinearities.The drawback is that traditional machine learning tends to require large number of data points hard to retrieve in many scientific and engineering fields like, for example, the field of battery discharge and degradation prediction. In this paper, we will present a hybrid modeling approach for tracking and forecasting battery aging based on ``as-used'' conditions.Our approach directly implements a reduced-order model based on Nerst and Butler-Volmer equations within a deep neural network framework.While most of the input-output relationship is captured by reduced-order models, the data-driven kernels reduce the gap between predictions and observations.The hybrid model estimates the overall battery discharge, and a multilayer perceptron models the battery internal voltage.Battery aging is characterized by time-dependent internal resistance and the amount of available Li-ions.We address the difficult issue of building and updating the aging model by reducing the need for reference discharge cycles.This is beneficial to operators, since it reduces the need of taking the batteries out of commission.We compensate for lack of reference discharge cycles by using a probabilistic model that leverages previously available fleet-wide information. We validate our approach using data publicly available through the NASA Prognostics Center of Excellence website.Results showed that our hybrid battery prognosis model can be successfully calibrated, even with a limited number of observations.Moreover, the model can help optimizing battery operation by offering long-term forecast of battery capacity.

The operation of complex systems can drift away from the initial design conditions, due to environmental conditions, equipment wear or specific restrictions. Steam generators are complex equipment and their proper operation relies on the identification of their most relevant parameters. An approach to rank the operational parameters of a subcritical steam generator of an actual 360 MW power plant is presented. An Artificial Neural Network - ANN delivers a model to estimate the steam generator efficiency, electric power generation and flue gas outlet temperature as a function of seven input parameters. The ANN is trained with a two-year long database, with training errors of 0.2015 and 0.2741 (mean absolute and square error) and validation errors of 0.32% and 2.350 (mean percent and square error). That ANN model is explored by means of a combination of situations proposed by a Design of Experiment - DoE approach. All seven controlled parameters showed to be relevant to express both steam generator efficiency and electric power generation, while primary air flow rate and speed of the dynamic classifier can be neglected to calculate flue gas temperature as they are not statistically significant. DoE also shows the prominence of the primary air pressure in respect to the steam generator efficiency, electric power generation and the coal mass flow rate for the calculation of the flue gas outlet temperature. The ANN and DoE combined methodology shows to be promising to enhance complex system efficiency and helpful whenever a biased behavior must be brought back to stable operation.

The Coffin–Manson–Basquin–Haford (CMBH) model is a well-accepted strain-life relationship to model fatigue life as a function of applied strain. In this paper, we propose a nonstationary uncertainty model for the CMBH model, alongside a Bayesian framework for model calibration and estimation of confidence and prediction intervals. Using Inconel 617 coupon test data, we compared our approach to traditional stationary variance models. The proposed uncertainty model successfully captures the fact that the variance of strain amplitude decreases with increasing fatigue life. Additionally, a discussion on how to use the proposed Bayesian framework to compensate for the lack of data by using prior information coming from similar alloys is also presented considering Hastealloy-X and Inconel 617 coupon data.

We present a physics-informed neural network modeling approach for missing physics estimation in cumulative damage models. This hybrid approach is designed to merge physics-informed and data-driven layers within deep neural networks. The result is a cumulative damage model in which physics-informed layers are used to model relatively well understood phenomena and data-driven layers account for hard-to-model physics. A numerical experiment is used to present the main features of the proposed framework. The test problem consists of predicting corrosion-fatigue of an Al 2024-T3 alloy used on panels of aircraft wings. Besides cyclic loading, panels are also subjected to saline corrosion. In this case, physics-informed layers implement the well-known Walker model for crack propagation, while data-driven layers are trained to compensate the bias in damage accumulation due to the corrosion effects. The physics-informed neural network is trained using full observation of inputs (far-field loads, stress ratio, and a corrosivity index defined per airport) and very limited observation of outputs (crack length at inspection for only a small portion of the fleet). Results show that the physics-informed neural network is able to learn how to compensate the missing physics of corrosion in the original fatigue model. Predictions from the hybrid model can be used in fleet management, for example, to prioritize inspection across the fleet or forecast ahead of time the number of planes with damage above a threshold.

Lithium-ion batteries are commonly used to power unmanned aircraft vehicles (UAVs). The ability to model and forecast remaining useful life of these batteries enables UAV reliability assurance. Building principled accurate models is challenging due to the complex electrochemistry that governs battery operation. Alternatively, reduced order models have the advantage of capturing the overall behavior of battery discharge, although they suffer from simplifications and residual discrepancy. This paper presents a hybrid modeling approach that directly implements physics within deep neural networks. While most of the input–output relationship is captured by reduced-order models, data-driven kernels reduce the gap between predictions and observations. A reduced-order model based on Nernst and Butler–Volmer equations represents the overall battery discharge, and a multilayer perceptron models the battery non-ideal voltage. Battery aging is characterized by time-dependent internal resistance and the amount of available Li-ions, which are modeled through an ensemble of variational Bayesian multilayer perceptrons. The approach is validated using data publicly available through the NASA Prognostics Center of Excellence website. Results showed that our hybrid battery prognosis model can be successfully calibrated, even with a limited number of observations. Moreover, the model can help optimizing battery operation by offering long-term forecast of battery capacity.

Prognosis and health management plays an important role in the control of costs associated with operating large industrial equipment, such as wind turbines and aircraft. It is only fair that engineers and scientists have vastly researched modeling approaches to support decision making. Motivated by the growing availability of data and computational power as well as the advances in algorithms and methods, modeling frameworks often merge elements of physics, machine learning, and statistical learning. In this paper, we present a review on modeling in support of prognosis and health management of industrial equipment. This survey complements the existing prognosis and health management literature by discussing how modeling strategies are influenced by industry-specific aspects such as maintenance approaches (e.g., reactive, proactive, and predictive), implementation factors (e.g., industry, business model, purpose, development, and deployment), as well as supporting technologies (sensing, repair, and modeling itself). We use the onshore wind energy and civil aviation industries to illustrate how these aforementioned aspects can influence modeling and implementation of prognosis and health management. The literature review is broad and covers contributions over the past 40 years. We close the paper with few topics that can motive research going forward.

In this paper, we present a novel physics-informed neural network modeling approach for corrosion-fatigue. The hybrid approach is designed to merge physics- informed and data-driven layers within deep neural networks. The result is a cumulative damage model where the physics-informed layers are used to model the relatively well understood physics (crack growth through Paris law) and the data-driven layers account for the hard to model effects (bias in damage accumulation due to corrosion). A numerical experiment is used to present the main features of the proposed physics-informed recurrent neural network for damage accumulation. The test problem consists of predicting corrosion-fatigue of an Al 2024-T3 alloy used on panels of aircraft wing. Besides cyclic loading, the panels are also subjected to saline corrosion. The physics-informed neural network is trained using full observation of inputs (far-field loads, stress ratio and a corrosivity index – defined per airport) and very limited observation of outputs (crack length at inspection for only a small portion of the fleet). Results show that the physics-informed neural network is able to learn the correction in the original fatigue model due to corrosion and predictions are accurate enough for ranking damage in different airplanes in the fleet (which can be used to prioritizing inspection).

Unexpected main bearing failure on a wind turbine causes unwanted maintenance and increased operation costs (mainly due to crane, parts, labor, and production loss). Unfortunately, historical data indicates that failure can happen far earlier than the design lives of the component. Root cause analysis investigations have pointed to problems inherent from manufacturing as the major contributor, as well as issues related to event loads (e.g., startups, shutdowns, and emergency stops), extreme environmental conditions, and maintenance practices, among others. Altogether, the multiple failure modes and contributors make modeling remaining useful life of main bearings a very daunting task. In this paper, we present a novel physics-informed neural network modeling approach for main bearing fatigue. The proposed approach is fully hybrid and designed to merge physics- informed and data-driven layers within deep neural networks. The result is a cumulative damage model where the physics-informed layers are used model the relatively well-understood physics (L10 fatigue life) and the data-driven layers account for the hard to model components (e.g., contribution due to poor greasing conditions).

The practice of applying strain-life relationships to model materials subjected to low cycle fatigue conditions has been widely-accepted for the past seventy years. The Coffin-Manson rule employs a double, two-parameter power law equation to correlate cycles to failure to strain range (or vice versa). Plastic strain range dominates low life, while elastic strain dominates high life. In some settings with well-established materials, copious amounts of test data are available. The median response and the Coffin-Manson parameters are determined through regression. Scatter in both strain amplitude and fatigue life values are a consequence of material and specimen variation, test technician attributes or more. Recent initiatives have endeavored to transition deterministic approaches over to probabilistic analogies. Confidence bands, lower ones in particular, are useful for understanding reliability curves and implementing reliability-based design and optimization. In settings with accelerated product development schedules, intentionally sparse sets of test data are used to play “what if” scenarios in the context of life prediction. We propose a non-stationary variance to model deviations from the Coffin-Manson rule and use Bayesian statistical methods to estimate model parameter uncertainties. The proposed approach is applied to the calibration of strain-life parameters of the candidate material Inconel 617, a Ni-base alloy used is combustion equipment. When compared with constant variance (such as in traditional regression), the results show an improved characterization of the confidence bounds for fatigue life. This is important as it indicates that the methodology can be used to manage the number of coupon test to achieve an acceptable convergence.

Services and warranties of large fleets of engineering assets is a very profitable business. The success of companies in that area is often related to predictive maintenance driven by advanced analytics. Therefore, accurate modeling, as a way to understand how the complex interactions between operating conditions and component capability define useful life, is key for services profitability. Unfortunately, building prognosis models for large fleets is a daunting task as factors such as duty cycle variation, harsh environments, inadequate maintenance, and problems with mass production can lead to large discrepancies between designed and observed useful lives. This paper introduces a novel physics-informed neural network approach to prognosis by extending recurrent neural networks to cumulative damage models. We propose a new recurrent neural network cell designed to merge physics-informed and data-driven layers. With that, engineers and scientists have the chance to use physics-informed layers to model parts that are well understood (e.g., fatigue crack growth) and use data-driven layers to model parts that are poorly characterized (e.g., internal loads). A simple numerical experiment is used to present the main features of the proposed physics-informed recurrent neural network for damage accumulation. The test problem consist of predicting fatigue crack length for a synthetic fleet of airplanes subject to different mission mixes. The model is trained using full observation inputs (far-field loads) and very limited observation of outputs (crack length at inspection for only a portion of the fleet). The results demonstrate that our proposed hybrid physics-informed recurrent neural network is able to accurately model fatigue crack growth even when the observed distribution of crack length does not match with the (unobservable) fleet distribution., Data and codes (including our implementation for both the multi-layer perceptron, the stress intensity and Paris law layers, the cumulative damage cell, as well as python driver scripts) used in this manuscript are publicly available on GitHub at https://github.com/PML-UCF/pinn. The data and code are released under the MIT License

A number of physical systems can be described by ordinary differential equations. When physics is well understood, the time dependent responses are easily obtained numerically. The particular numerical method used for integration depends on the application. Unfortunately, when physics is not fully understood, the discrepancies between predictions and observed responses can be large and unacceptable. In this paper, we propose an approach that uses observed data to estimate the missing physics in the original model (i.e., model-form uncertainty). In our approach, we first design recurrent neural networks to perform numerical integration of the ordinary differential equations. Then, we implement the recurrent neural network as a directed graph. This way, the nodes in the graph represent the physics-informed kernels found in the ordinary differential equations. We quantify the missing physics by carefully introducing data-driven in the directed graph. This allows us to estimate the missing physics (discrepancy term) even for hidden nodes of the graph. We studied the performance of our proposed approach with the aid of three case studies (fatigue crack growth, corrosion-fatigue crack growth, and bearing fatigue) and state-of-the-art machine learning software packages. Our results demonstrate the ability to perform estimation of discrepancy, reducing gap between predictions and observations, at reasonable computational cost.

Field failures of wind turbine main bearings yield to undesired downtime and significant maintenance costs. Fatigue failure is a dominant mode for legacy turbines, which can be expressed with physics-informed models to some extent. However, these models often inherent large uncertainties due to unknown lubricant degradation mechanism. Therefore, periodical assessment of the grease state plays a crucial role in calibration of bearing fatigue models. As opposed to detailed laboratory analysis, grease visual inspection can lead to large uncertainties in characterization of grease condition (although visual inspection can be cost and time effective). In this paper, we introduce a hybrid model for main bearing fatigue damage accumulation and calibrated using only visual grease inspections. In our hybrid model, bearing fatigue damage portion consists of known physics formulations, and unknown grease degradation is represented with deep neural networks. In addition, we introduce a custom tailored classifier that enables the model to map from damage scale to visual rankings. Results showed that the bearing fatigue prognosis model can be successfully calibrated, even with limited and noisy field observations. Moreover, the model can help optimizing park reliability by suggesting turbine-specific regreasing intervals. The source codes and links to the data can be found in the following GitHub repository https://github.com/PML-UCF/pinn_wind_bearing .

In the service market targeting fleets of industrial assets (e.g., aircraft, jet engines, wind turbines, etc.), original equipment manufacturers and service providers compete with one another through offers covering day-to-day service as well as major maintenance and repairs over. Since decision-making is highly guided by reliability models, it is safe to say that services profitability dependents on the ability to understand the complex stochastic interactions between operating conditions and component capability. Unfortunately, factors such as aggressive mission mixes introduced by operators, exposure to a harsh environment, inadequate maintenance, and problems with mass production can lead to large discrepancies between predicted and observed useful lives. This paper focuses on the quantification of the infant mortality impact on fleets of industrial assets. A numerical experiment is used to study how the number of failure observations and fleet size impacts the modeling of fleet reliability. Dynamic Bayesian networks implementing physics-based models are used to model fleet unreliability considering the effects of bad batch of materials.The results demonstrate that material capability, penetration of bad batch of material in the fleet, and fleet size drastically influence the model accuracy. Therefore, small fleet operators, which naturally observe a lownumber of failures, have to deal with larger uncertainties when quantifying infant mortality. This negatively impacts their ability to allocate resources such as inventory, labor, and account for the loss of productivity while servicing their fleet. With large fleet operators, on the other hand, large number of failure observations can cause high financial burden. Nevertheless, it also allows for reduced uncertainty in building/updating the reliability models, which can help their ability to forecast future failures and make provisions for service and maintenance. Finally, the results also show that measures such as recommissioning of the fleet and inspection campaigns can mitigate the effects of fleet-wide early life problems.

Available historical field data shows that wind turbine main bearing failure can lead to major operation and maintenance costs due to unscheduled downtime. For legacy turbines, fa- tigue is one of the major failure modes and, to a degree, can be partially modeled with physics-based formulations. Unfor- tunately, existing bearing fatigue models can potentially be inaccurate due to lack of understanding of the lubricant degra- dation. One way to enhance these models is to track the grease damage along with the bearing fatigue damage. However, the need of grease degradation data can become an impedi- ment for such strategy. In this paper, we will demonstrate that it is possible to calibrate grease degradation models with cost-efficient periodic visual inspections. Knowing that such inspections introduce observation uncertainty to the model, we will use a hybrid physics-informed deep neural networks to quantify such uncertainties within our models. We built a hybrid model that fuses the physics-based understanding of the bearing fatigue failure with the ability of data-driven layers to compensate the missing physics, with respect to the grease degradation. The proposed hybrid model is also ca- pable of decoding uncertain visual grease inspections with a custom designed classifier. We illustrate the merits of the model with the support of case studies, where we test inspec- tion with different levels of conservatism to train the model and compare the predictions of these models on an artificial wind park. Results from the case studies indicate the success- ful prognostic performance of the trained with limited and noisy observations. While grease damage is predicted with 0.3% root mean square error as a result of baseline inspection campaign, bearing life is prediction is conservatively off only by months for aggressive turbines that have 10 years of life.

We present a tutorial on how to directly implement integration of ordinary differential equations through recurrent neural networks using Python. In order to simplify the implementation, we leveraged modern machine learning frameworks such as TensorFlow and Keras. Besides, offering implementation of basic models (such as multilayer perceptrons and recurrent neural networks) and optimization methods, these frameworks offer powerful automatic differentiation. With all that, the main advantage of our approach is that one can implement hybrid models combining physics-informed and data-driven kernels, where data-driven kernels are used to reduce the gap between predictions and observations. Alternatively, we can also perform model parameter identification. In order to illustrate our approach, we used two case studies. The first one consisted of performing fatigue crack growth integration through Euler’s forward method using a hybrid model combining a data-driven stress intensity range model with a physics-based crack length increment model. The second case study consisted of performing model parameter identification of a dynamic two-degree-of-freedom system through Runge–Kutta integration. The examples presented here as well as source codes are all open-source under the GitHub repository https://github.com/PML-UCF/pinn_code_tutorial .

Services and warranties of large fleets of engineering assets is a very profitable business where original equipment manufacturers and independent service providers offer contracts designed to cover events in day-to-day service as well as major maintenance and repairs over the life of the asset. Accurate reliability modeling, as a way to understand how the complex stochastic interactions between operating conditions and component capability define useful life, is key for services profitability. The modeling task is daunting as factors such as aggressive mission mixes introduced by operators, exposure to harsh environment, inadequate maintenance, and problems with mass production (bad batch of materials) can lead to large discrepancies between designed and observed useful lives. This paper is focused on how to quantify the impact of infant mortality in fleets of industrial assets. A simple numerical experiment is used to address the fundamental question: how does number of observations and fleet size interact with each other in fleet management? The results demonstrate that material capability, penetration of bad batch of material in the fleet, and commissioning time can drastically influence fleet unreliability. Moreover, infant mortality due to manufacturing problems/material capability is a manifestation of an outlier problem. As a consequence, the propensity to observe first failures depend on the actual fleet size. Since failure observations are used to build/update the reliability models, small fleet operators have to deal with large uncertainties when quantifying infant mortality. This impacts their ability to make provisions for service and maintenance (inventory, labor, loss of productivity, etc.). Although the large number of failure observations causes a financial burden in large fleet operators, it also allows for reduced uncertainty in building/updating the reliability models. In turn, this improves their ability to forecast future failures and make provisions for service and maintenance.

The growing power of computers enabled techniques created for design and analysis of simulations to be applied to a large spectrum of problems and to reach high level of acceptance among practitioners. Generally, when simulations are time consuming, a surrogate model replaces the computer code in further studies (e.g., optimization, sensitivity analysis, etc.). The first step for a successful surrogate modeling and statistical analysis is the planning of the input configuration that is used to exercise the simulation code. Among the strategies devised for computer experiments, Latin hypercube designs have become particularly popular. This paper provides a tutorial on Latin hypercube design of experiments, highlighting potential reasons of its widespread use. The discussion starts with the early developments in optimization of the point selection and goes all the way to the pitfalls of the indiscriminate use of Latin hypercube designs. Final thoughts are given on opportunities for future research. Copyright © 2015 John Wiley & Sons, Ltd.