Model-Free Inverse H-Infinity Control for Imitation Learning

This paper proposes a data-driven model-free inverse reinforcement learning (IRL) algorithm tailored for solving an inverse H-infinity control problem. In the problem, both an expert and a learner engage in H-infinity control to reject disturbances and the learner's objective is to imitate the expert's behavior by reconstructing the expert's performance function through IRL techniques. Introducing zero-sum game principles, we first formulate a model-based single-loop IRL policy iteration algorithm that includes three key steps: updating the policy, action, and performance function using a new correction formula and the standard inverse optimal control principles. Building upon the model-based approach, we propose a model-free single-loop off-policy IRL algorithm that eliminates the need for initial stabilizing policies and prior knowledge of the dynamics of expert and learner. Also, we provide rigorous proof of convergence, stability, and Nash optimality to guarantee the effectiveness and reliability of the proposed algorithms. Furthermore, we show-case the efficiency of our algorithm through simulations and experiments, highlighting its advantages compared to the existing methods.
NSFC [61991404, 62394342, U22A2049]; Liaoning Revitalization Talents Program [XLYC2007135]; Science and Technology Major Project of Liaoning Province [2020JH1, 10100008]; Key Research and Development Program of Liaoning Province [2023JH26, 10200011]; Research Program of the Liaoning Liaohe Laboratory [LLL23ZZ-05-01]
This work was supported in part by NSFC under Grant 61991404, Grant 62394342, and Grant U22A2049; in part by Liaoning Revitalization Talents Program under Grant XLYC2007135; in part by the 2020 Science and Technology Major Project of Liaoning Province under Grant 2020JH1/10100008; in part by the Key Research and Development Program of Liaoning Province under Grant 2023JH26/10200011; and in part by the Research Program of the Liaoning Liaohe Laboratory under Grant LLL23ZZ-05-01.