Echo state network-based online optimal control for discrete-time nonlinear systems

This paper investigates the online optimal control problem of discrete-time nonlinear systems using echo state network (ESN)-based adaptive dynamic programming (ADP) method. An online iterative learning algorithm is proposed to solve the partial differential Hamilton–Jacobi–Bellman (HJB) equation in real time. A novel neural networks (NN) critic-actor architecture is presented using two ESNs to implement the ADP method. Then, two online learning laws of the output weights are designed for searching the optimal cost function and control policy. The stability of system and output weights is analysed using Lyapunov approach. Three simulations are given to show the feasibility and effectiveness of the designed algorithm.