Echo state network-based online optimal control for discrete-time nonlinear systems

Chong Liu; Huaguang Zhang; Yanhong Luo; Kun Zhang

doi:10.1016/j.amc.2021.126324

Echo state network-based online optimal control for discrete-time nonlinear systems

Chong Liu
, Huaguang Zhang^*
, Yanhong Luo
, Kun Zhang

^*Corresponding author for this work

Northeastern University China

Research output: Contribution to journal › Article › peer-review

11 Scopus citations

Abstract

This paper investigates the online optimal control problem of discrete-time nonlinear systems using echo state network (ESN)-based adaptive dynamic programming (ADP) method. An online iterative learning algorithm is proposed to solve the partial differential Hamilton–Jacobi–Bellman (HJB) equation in real time. A novel neural networks (NN) critic-actor architecture is presented using two ESNs to implement the ADP method. Then, two online learning laws of the output weights are designed for searching the optimal cost function and control policy. The stability of system and output weights is analysed using Lyapunov approach. Three simulations are given to show the feasibility and effectiveness of the designed algorithm.

Original language	English
Article number	126324
Journal	Applied Mathematics and Computation
Volume	409
DOIs	https://doi.org/10.1016/j.amc.2021.126324
State	Published - 15 Nov 2021
Externally published	Yes

Keywords

Adaptive dynamic programming (ADP)
Discrete-time systems
Echo state network (ESN)
Optimal control

Access to Document

10.1016/j.amc.2021.126324

Cite this

@article{2e859986e93f42d0bfd676a1cb17d363,

title = "Echo state network-based online optimal control for discrete-time nonlinear systems",

abstract = "This paper investigates the online optimal control problem of discrete-time nonlinear systems using echo state network (ESN)-based adaptive dynamic programming (ADP) method. An online iterative learning algorithm is proposed to solve the partial differential Hamilton–Jacobi–Bellman (HJB) equation in real time. A novel neural networks (NN) critic-actor architecture is presented using two ESNs to implement the ADP method. Then, two online learning laws of the output weights are designed for searching the optimal cost function and control policy. The stability of system and output weights is analysed using Lyapunov approach. Three simulations are given to show the feasibility and effectiveness of the designed algorithm.",

keywords = "Adaptive dynamic programming (ADP), Discrete-time systems, Echo state network (ESN), Optimal control",

author = "Chong Liu and Huaguang Zhang and Yanhong Luo and Kun Zhang",

note = "Publisher Copyright: {\textcopyright} 2021",

year = "2021",

month = nov,

day = "15",

doi = "10.1016/j.amc.2021.126324",

language = "英语",

volume = "409",

journal = "Applied Mathematics and Computation",

issn = "0096-3003",

publisher = "Elsevier Inc.",

}

TY - JOUR

T1 - Echo state network-based online optimal control for discrete-time nonlinear systems

AU - Liu, Chong

AU - Zhang, Huaguang

AU - Luo, Yanhong

AU - Zhang, Kun

PY - 2021/11/15

Y1 - 2021/11/15

N2 - This paper investigates the online optimal control problem of discrete-time nonlinear systems using echo state network (ESN)-based adaptive dynamic programming (ADP) method. An online iterative learning algorithm is proposed to solve the partial differential Hamilton–Jacobi–Bellman (HJB) equation in real time. A novel neural networks (NN) critic-actor architecture is presented using two ESNs to implement the ADP method. Then, two online learning laws of the output weights are designed for searching the optimal cost function and control policy. The stability of system and output weights is analysed using Lyapunov approach. Three simulations are given to show the feasibility and effectiveness of the designed algorithm.

AB - This paper investigates the online optimal control problem of discrete-time nonlinear systems using echo state network (ESN)-based adaptive dynamic programming (ADP) method. An online iterative learning algorithm is proposed to solve the partial differential Hamilton–Jacobi–Bellman (HJB) equation in real time. A novel neural networks (NN) critic-actor architecture is presented using two ESNs to implement the ADP method. Then, two online learning laws of the output weights are designed for searching the optimal cost function and control policy. The stability of system and output weights is analysed using Lyapunov approach. Three simulations are given to show the feasibility and effectiveness of the designed algorithm.

KW - Adaptive dynamic programming (ADP)

KW - Discrete-time systems

KW - Echo state network (ESN)

KW - Optimal control

UR - https://www.scopus.com/pages/publications/85107661284

U2 - 10.1016/j.amc.2021.126324

DO - 10.1016/j.amc.2021.126324

M3 - 文章

AN - SCOPUS:85107661284

SN - 0096-3003

VL - 409

JO - Applied Mathematics and Computation

JF - Applied Mathematics and Computation

M1 - 126324

ER -

Echo state network-based online optimal control for discrete-time nonlinear systems

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this