Leader-Follower Formation Control for Fixed-Wing UAVs using Deep Reinforcement Learning

This paper studies a fundamental formation flight scenario for fixed-wing unmanned aerial vehicles (UAVs) based on the leader-follower guidance and control frame using deep reinforcement learning (DRL) method. Firstly, on the basis of typical path following guidance problem, this paper proposes a complete dynamics for fixed-wing vehicle formation tracking flight with both acceleration and angular rate control. The tracking error dynamics with respect to the Serret- Frenet frame is derived where the singularity problem is avoided. Secondly, DRL methods are further introduced to cope with the highly coupled nonlinear problem. Based on both original application and indirect modifications of error dynamics, the online learning environments are respectively constructed. Thirdly, the implementation and comparative analysis of both deep deterministic policy gradient (DDPG) and deep Q-network (DQN) methods for solving the formation control problem are provided using deep neural network (DNN) approximation. Finally, the learning and control results of both different models and diverse DRL methods are given to verify the efficiency and applicability.