Search

We discuss certain basic features of the equation-free (EF) approach to modeling and computation for complex/multiscale systems. We focus on links between the equation-free approach and tools from systems and control theory (design of experiments, data analysis, estimation, identification and feedback). As our illustrative example, we choose a specific numerical task (the detection of stability boundaries in parameter space) for stochastic models of two simplified heterogeneous catalytic reaction mechanisms. In the equation-free framework the stochastic simulator is treated as an experiment (albeit a computational one). Short bursts of fine scale simulation (short computational experiments) are designed, executed, and their outputs processed and fed back to the process, in integrated protocols aimed at performing the particular coarse-grained task (the detection of a macroscopic instability). Two distinct approaches are presented; one is a direct translation of our previous protocol for adaptive detection of instabilities in laboratory experiments [Rico-Martinez, R., Krisher, K., Flatgen, G., Anderson, J. S., & Kevrekidis, I. G. (2003). Adaptive detection of instabilities: An experimental feasibility study. Physica D, 176, 1–18]; the second approach is motivated from numerical bifurcation algorithms for critical point detection. A comparison of the two approaches brings forth a key feature of equation-free computation: computational experiments can be easily initialized at will, in contrast to laboratory ones.

We present and discuss an inherent shortcoming of neural networks used as discrete-time models in system identification, time series processing, and prediction. Trajectories of nonlinear ordinary differential equations (ODEs) can, under reasonable assumptions, be integrated uniquely backward in time. Discrete-time neural network mappings derived from time series, on the other hand, can give rise to multiple trajectories when followed backward in time: they are in principle noninvertible. This fundamental difference can lead to model predictions that are not only slightly quantitatively different, but qualitatively inconsistent with continuous time series. We discuss how noninvertibility arises, present key analytical concepts and some of its phenomenology. Using two illustrative examples (one experimental and one computational), we demonstrate when noninvertibility becomes an important factor in the validity of artificial neural network (ANN) predictions, and show some of the overall complexity of the predicted pathological dynamical behavior. These concepts can be used to probe the validity of ANN time series models, as well as provide guidelines for the acquisition of additional training data.

Advances in scientific computation and developments in spatially resolved sensor technology have, in recent years, critically enhanced our ability to develop modeling strategies and experimental techniques for the study of the spatiotemporal response of distributed nonlinear systems. The usual alternatives for the modeling of these systems, simplifying techniques that seek to capture the distributed system dynamics through lumped parameter models, can be drastically underresolved, and miss important features of the true system response. Robust implementations of distributed system identification algorithms based on detailed spatiotemporal experimental data have, therefore, an important role to play. In this contribution we present a methodology for the identification of distributed parameter systems, based on artificial neural network architectures, motivated by standard numerical discretization techniques used for the solution of partial differential equations.

The success of nonlinear system identification and characterization from experimental time-series often depends on the appropriate pre-processing of the data. This pre-processing can, in many instances, be achieved through linear and/or “nonlinear” principal component analysis. In recent years, neural network based techniques as a means to perform the nonlinear principal component (NLPC) analysis have gained increased attention. In this contribution, we address an inherent shortcoming of these techniques: the self-consistency problem. We present a modification to the usual feedforward neural network architecture used to extract the NLPCs that results in attenuation of this problem. The proposed modification may significantly reduce representation errors in many other applications for which NLPC analysis is a powerful tool, such as feature extraction and image processing.

Artificial Neural Networks (ANNs) can be used for grey-box or black-box modeling of continuous-time systems by placing them in a framework based on numerical integration techniques. When an implicit integration scheme is used as a template, it imposes a recurrent structure on the overall network. Here we present three algorithms suitable for the training of such “network-plus-integrator” assemblies and compare their relative computational efficiencies. Pineda's Recurrent Back-Propagation (RBP) training method is recast to exploit the structure of the assembly. The second approach is RBP modified to evaluate partial derivatives of network outputs with respect to parameters exactly, while the third is a Newton-Raphson based algorithm in which outputs of the network and partial derivatives are computed at each step instead of approximated. We compare the methods via an illustrative example and discuss aspects of training in a parallel computing environment.