Search

The basic problem considered here is the optimal control of finite-state machines operating over an infinite time interval. This is essentially a deterministic version of the theories of Howard [4] and Eaton and Zadah [5] on the control of Markov [email protected]? The deterministic theory presents certain special problems, not found in the stochastic theory. This is especially so for finite state machines whose closed-loop dynamics always include periodic motions, cyclic machines. This class of dynamical system is of interest in Economic and Biological systems where the system behavior is often inherently periodic. Necessary and sufficient conditions are given for optimal control. However, the necessity condition, given in Theorem 4, requires more restrictive assumptions than the sufficiency condition, given in Theorem 2. Also, while both cyclic and absorbing-state machines, with all transient states and one absorbing state, are considered, the major emphasis is on cyclic machines.

Publisher Summary This chapter discusses the basic concepts related to optimality, insensitivity, and game theory. The problem of sensitivity is always critical in the design of engineering systems. In the case of optimal control systems, however, the problem takes on special importance as these systems often tend to be tuned” with respect to plant parameter values, which in many cases are at best only approximately known. Most optimization techniques completely ignore the sensitivity problem and, conversely, most sensitivity studies completely ignore the optimization problem. The chapter presents an attempt to formulate the insensitivity problem in a general way. It explains briefly and qualitatively some of the notions of game theory, and how they enter the insensitivity problem, and reviews those theorems which strike as particularly relevant to that problem. An account of the special features imposed on the game-theoretical ideas by the context of control theory is also presented in the chapter.

In this paper the optimal discrete-time linear-quadratic regulator problem is carefully presented and the basic results are reviewed. Dynamic programming is used to determine the optimization equations. Special attention is given to problems unique to the discrete-time case; this includes, for example, the possibility of a singular system matrix and a singular control-effort weighting matrix. Some problems associated with sampled-data systems are also summarized, e.g., sensitivity to sampling time, and loss of controllability due to sampling. Computational methods for the solution of the optimization equations are outlined and a simple example is included to illustrate the various computational approaches.

This study considers the problem of determining optimal feedback control laws for linear stochastic systems with amplitude-constrained control inputs. Two basic performance indices are considered, average time and average integral quadratic form. The optimization interval is random and defined as the first time a trajectory reaches the terminal region R . The plant is modeled as a stochastic differential equation with an additive Wiener noise disturbance. The variance parameter of the Wiener noise process is assumed to be suitably small. A singular perturbation technique is presented for the solution of the stochastic optimization equations (second-order partial differential equation). A method for generating switching curves for the resulting optimal bang-bang control system is then developed. The results are applied to various problems associated with a second-order purely inertial system with additive noise at the control input. This problem is typical of satellite attitude control problems.

An optimal stochastic control problem is considered with a probability criterion. A stochastic differential equation model is assumed. The optimization problem is to maximize the probability that the state trajectory remain in a given bounded region D over a given finite time interval. This type of criterion is especially relevant to certain technical problems where it is essential that certain state variables not exceed given values, and is closely related to the concepts of finite time stability. For the class of dynamical systems considered, the optimal control is bang-bang. A numerical method developed by Samarskii is used to solve the optimization equation, a nonlinear partial differential equation of the parabolic type. A second-order system is computed to illustrate numerical results. Time-varying switching curves for the optimal bang-bang solution are plotted.

This paper derives, via Pontryagin's minimum principle, the invariance of the sensitivity of cost for optimal systems with respect to open-loop and closed-loop control law implementation when system parameters undergo small variations. The result obtained here is more general than that of Pagurek[1] and somewhat easier to apply than that of Witsenhausen.[2] The cost of the sensitivity vector is simply related to the gradient of the Hamiltonian function.

The theory of games is applied to the design of systems with unknown plant parameters. It is assumed that a controller structure is known and furthermore that this controller is optimal when the controller parameters are equal to the plant parameters. The performance index then becomes a function of plant and controller parameters. This function is treated as a pay-off function with the antagonists represented by the controller and plant parameters. The theory is illustrated with some simple samples.