AUV Path Planning with Kinematic Constraints in Unknown Environment Using Reinforcement Learning

Path planning has been considered as a challenging and critical issue in the management of autonomous underwater vehicle (AUV) systems. Although in the past years, the study of AUV path planning has obtained numerous achievements, many researches only considered the optimal path in a known grid-based environment, and ignored nonlinear kinematic characteristics of AUVs, which is unrealistic. In this paper, a reinforcement learning based path planning algorithm is proposed, with the nonlinear kinematic constraints of the AUV in unknown continuous environments. The proposed algorithm utilizes the sonar array to detect the randomly placed obstacles and plans a collision-free path that connects the start and target points even the map data is not known in advance. Extensive analysis and simulations are conducted in unknown continuous environments, and verify the validity and efficiency of the proposed algorithm.