A direct search for time-optimal control in stochastic systems †

A feedback control policy is formulated for minimization of the expected length of time required for the state of a stochastic dynamic system to first reach a desired terminal region. The control variables are constrained and it is assumed that noise-free observations of the system state are available at all times. Dynamic programming leads to a non-linear partial differential equation that is very difficult to solve. Instead the difference between the stochastic and the more tractable deterministic equations for the switching hyperplane is represented by a truncated power series, and the optimal switching hyperplane for a ‘ bang-bang ’ controller is then computed through a search on the power series coefficients using repetitive simulations. Numerical results for the location of the stochastic switching curves for a specific second-order system with additive gaussian white noise are presented and discussed.