Heuristic dynamic programming algorithm for optimal control design of linear continuous-time hyperbolic PDE systems

This work considers the optimal control problem of linear continuous-time hyperbolic partial differential equation (PDE) systems with partially unknown system dynamics. To respect the infinite-dimensional nature of the hyperbolic PDE system, the problem can be reduced to finding a solution of the space-dependent Riccati differential equation (SDRDE), which requires the full system model. Therefore, a heuristic dynamic programming (HDP) algorithm is proposed to achieve online optimal control of the hyperbolic PDE system, which online collects data accrued along system trajectories and learns the solution of the SDRDE without requiring the internal system dynamics. The convergence of HDP algorithm is established by showing that the HDP algorithm generates a nondecreasing sequence which uniformly converges to the solution of the SDRDE. For implementation purposes, the HDP algorithm is realized by developing an approximate approach based on the method of weighted residuals. Finally, the application on a steam-jacketed tubular heat exchanger demonstrates the effectiveness of the developed control approach.