Search

Summary: In this paper, we develop an online learning algorithm for solving the Bellman equation for affine in the control discrete‐time nonlinear uncertain dynamical systems. To ensure accelerated learning of our algorithm in generating optimal control policies, we use an actor‐critic structure predicated on higher‐order tuner laws. More specifically, we construct a Nesterov‐like architecture involving momentum‐based learning laws leading to an accelerated convergence of the optimal control policy. The proposed online learning‐based optimal control framework guarantees uniform ultimate boundedness of the closed‐loop system under the assumption that the system is persistently excited. Finally, two illustrative numerical examples are provided to demonstrate the efficacy of the proposed approach. [ABSTRACT FROM AUTHOR]

We consider the problem of optimal control of a linear time-invariant discrete-time system by inaccurate measurements of its output signals subject to guaranteed satisfaction of geometric constraints on the output signals. We study the case in which a minimal realization of the system in the state space is known and the case where the parametric model of the system is not known. A novel method is proposed for solving the problem in the case of an unknown model. The method is based on a single observed trajectory of the input and output signals of the system and allows omitting the stage of parametric identification of the system. [ABSTRACT FROM AUTHOR]

For systems that can only be locally stabilized, control laws and their effective regions are both important. In this paper, invariant policy iteration is proposed to solve the optimal control of discrete-time systems. At each iteration, a given policy is evaluated in its invariantly admissible region, and a new policy and a new region are updated for the next iteration. Theoretical analysis shows the method is regionally convergent to the optimal value and the optimal policy. Combined with sum-of-squares polynomials, the method is able to achieve the near-optimal control of a class of discrete-time systems. An invariant adaptive dynamic programming algorithm is developed to extend the method to scenarios where system dynamics is not available. Online data are utilized to learn the near-optimal policy and the invariantly admissible region. Simulated experiments verify the effectiveness of our method. [ABSTRACT FROM AUTHOR]

In this paper, an $H_{\infty }$ optimal tracking controller for completely unknown discrete-time nonlinear systems with control constraints is obtained by using an iterative adaptive learning algorithm. An augmented system is established by integrating the tracking error system and the reference trajectory. As an identifier of the unknown systems, a neural network (NN) is introduced with asymptotic stability of the estimation error. An action–disturbance–critic NN structure is proposed to implement the iterative dual heuristic programming algorithm with convergence guarantee of the costate function and the control policy. Simulation results and comparisons are provided to illustrate the superior performance of the designed optimal tracking controller. [ABSTRACT FROM AUTHOR]

This article investigates the two paradigms of norm optimal iterative learning control (NOILC) and parameter optimal iterative learning control (POILC) for multivariable (MIMO) ℓ-input, m-output linear discrete-time systems. The main result is a proof that, despite their algebraic and conceptual differences, they can be unified using linear quadratic multi-parameter optimisation techniques. In particular, whilst POILC has been naturally regarded as an approximation to NOILC, it is shown that the NOILC control law can be generated from a suitable choice of control law parameterisation and objective function in a multi-parameter MIMO POILC problem. The form of this equivalence is used to propose a new general approach to the construction of POILC problems for MIMO systems that approximates the solution of a given NOILC problem. An infinite number of such approximations exist. This great diversity is illustrated by the derivation of new convergent algorithms based on time interval and gradient partition that extend previously published work. [ABSTRACT FROM PUBLISHER]