Space-time marching-based neural direct method for the high-dimensional Hamilton-Jacobi-Bellman equation

The Hamilton-Jacobi-Bellman (HJB) equation is a fundamental partial differential equation (PDE) in optimal control theory, but solving it in a high-dimensional space presents severe challenges due to the curse of dimensionality (CoD). Recently, a class of methods that combine neural networks with traditional direct methods, referred to here as neural direct methods, has achieved success in the numerical solution of the high-dimensional HJB equation. Nevertheless, existing neural direct methods struggle with convergence when applied to the HJB equation with a long time horizon. A similar issue also arises when the effective solution domains of these methods are expanded. Through theoretical analysis, we identify one underlying cause of these issues as the exponential propagation of nonlinearity in the state dynamics over time, which significantly increases the difficulty of solving long-horizon problems. To address these challenges, we propose a space-time marching-based neural direct method (STM-NDM), which solves the HJB equation with gradually increasing difficulty levels. For each level, the HJB equation is recast as an optimal feedback control problem that accounts for midway disturbances, which is solved by a modified direct single shooting (DSS) method with neural network parameterization. Experimental results on several high-dimensional HJB equations corresponding to path planning problems validate the effectiveness of the STM-NDM.