Tracking control optimization scheme of continuous-time nonlinear system via online single network adaptive critic design method

In this paper, the optimal tracking control problem (OTCP) for a class of continuous-time nonlinear systems with infinite horizon cost is discussed. An online adaptive critic design method is proposed to learn the solution of OTCP by constructing an augmented system associated with a discounted performance function, which is composed of the tracking errors and reference trajectory dynamics. Only one neural network (NN) is used as critic module for approximating the performance function in the solution procedure, and thus the architecture is simpler than the typical action-critic structure, which needs more computational load from neural networks. Therefore, by the means of the approximate policy iteration, the tracking errors get converged to a region near zero and the parameters of critic module get converged to the optimal ones based on our proposed method. Both the convergence of the NN weights and the stability of the tracking error dynamics are guaranteed by the Lyapunov theory. Two simulation examples are proposed to verify the effectiveness of the proposed method.