Linear quadratic tracking control of unknown discrete-time systems using value iteration algorithm

In this paper, an optimal tracking control scheme is proposed to solve the infinite-horizon linear quadratic tracking (LQT) problem using iterative adaptive dynamic programming (ADP) algorithm. The reference trajectory is assumed to be produced by a linear command generator. First, via system transformation, an augmented system composed of controlled system and command generator is constructed. Then we derive the Bellman equation in terms of the transformed system with discount factor in cost function. In order to avoid requirement for knowledge of system dynamics, the iterative ADP algorithm is introduced to solve the Bellman equation with convergence analysis. A novel approach based on controllability and observability analysis is presented to show the stability of tracking error. For facilitating the implementation of this iterative approach, three neural networks (NNs) are employed as parametric structures to identify the unknown system dynamics, approximate performance function and search control policy, respectively. Finally, a simulation example is included to verify the effectiveness of the proposed scheme.