Adaptive dynamic programming for optimal control of nonlinear distributed parameter systems

In this chapter, we consider the optimal control problem of distributed parameter systems (DPSs), which is described by general highly dissipative nonlinear partial differential equations (PDEs). Initially, Karhunen-Loève decomposition is employed to compute empirical eigenfunctions of the DPS based on the method of snapshots. These empirical eigenfunctions together with the singular perturbation technique are then used to obtain a finite-dimensional slow subsystem of ordinary differential equations that accurately describes the dominant dynamics of the DPS. Then, the optimal control problem is reformulated on the basis of the slow subsystem, which is further converted to solve a Hamilton-Jacobi-Bellman equation (HJBE). The HJBE is a nonlinear PDE that has been proven to be impossible to solve analytically. Thus, two adaptive dynamic programming methods are developed to solve the HJBE. First, an adaptive optimal control method based on neurodynamic programming is developed by using the neural network for approximating the value function and an online weight tuning law is proposed without requiring an initial stabilizing control policy. Moreover, by considering the DPS to be partially unknown, an adaptive control approach based on policy iteration is developed by using online system data. Through the simulation studies on the nonlinear diffusion-convection-reaction process, the results demonstrate the efficiency of the developed methods.