RL-Based Optimal Low-Altitude Trajectory Tracking for UAVs with Finite-Time Weight Convergence

Unmanned aerial vehicles (UAVs) hold significant promise for a wide range of low-altitude applications, such as search and rescue, payload delivery, and surveillance. In such critical scenarios, achieving accurate trajectory tracking with minimal control effort and robust disturbance rejection is challenging due to system nonlinearities, external disturbances, and parameter uncertainties. This paper proposes a reinforcement learning (RL)-based optimal trajectory tracking method with disturbance rejection for UAVs to balance tracking performance and control effort. Firstly, by leveraging fundamental filtering operations and the invariant manifold principle, an unknown system dynamics estimator (USDE)-based steady-state controller is designed to ensure robust tracking performance. Second, to achieve optimization capability, an approximate optimal regulator is established by virtue of single-critic neural network to stabilize error dynamics and minimize value function. Specifically, by fully utilizing the historical data and current information, we propose a finite-time learning policy driven by neural network weight errors to approximate the Hamilton-Jacobi-Bellman (HJB) solutions with faster convergence rate. Simulation results validate the effectiveness and superior performance of the proposed method.