Output-feedback Q-learning for discrete-time linear H^∞ tracking control: A Stackelberg game approach

In this article, an output-feedback Q-learning algorithm is proposed for the discrete-time linear system to deal with the (Formula presented.) tracking control problem. The problem is formulated as a zero-sum game in the Stackelberg game framework with a discount factor to make the value function bounded. According to the principle of optimality, the game algebraic Riccati equation (GARE) is derived and solved by the Q-learning algorithm to get the optimal solution of the Stackelberg game without requiring the knowledge of system dynamics and state. It is proved that the solution of the algorithm converges to the optimal control input and the worst-case disturbance with excitation noises during training, and the Stackelberg strategy can achieve a lower (Formula presented.) disturbance attenuation level than the Nash one. Moreover, the impacts of the discount factor on the stability of the closed-loop system and solvability of the GARE are analyzed to provide some criteria for the choice of the discount factor. Simulation examples are provided to validate the effectiveness of the algorithm.