Adaptive Path-Tracking Controller Embedded With Reinforcement Learning and Preview Model for Autonomous Driving

Path tracking control is a crucial function in an autonomous vehicle. Previous studies have presented conventional model-based controllers, which lack generalization ability, especially in dealing with high-curvature cases. In contrast to those model-based controllers, data-driven controllers, represented by reinforcement learning (RL), are promising to improve tracking control accuracy and generalization by online learning. However, the inherent inexplicability of RL degrades the stability of an RL-based controller, further limiting its applications in real-world autonomous driving cases. To resolve this issue, this study proposes an adaptive tracking controller based on RL, allowing to enhance network stability through a preview model and adaptive correction. It is composed of three modules, namely a lateral controller, a lateral adaptive corrector, and a longitudinal speed planner. Specifically, the preview control theory is combined with the twin delayed deep deterministic policy gradient (TD3) method to improve the tracking convergence of the lateral controller; the adaptive corrector derives correction compensations based on predicted waypoints along a path, thus enhancing control accuracy; the longitudinal planner outputs a speed profile via a reward increment soft actor-critic algorithm, through which large lateral tracking errors are reduced. Using an intelligent vehicle with sufficient steering capability, simulations and real-world tests show that curvy paths with a curvature larger than 2.0 m⁻¹ can be closely tracked within an error bound of 0.1 m.