Small UAV Formation Control in Complex Environments Based on Deep Reinforcement Learning

In addressing the problem of multi-UAV formation flight in complex environments, we propose a novel formation control framework for small fixed-wing UAVs. The framework integrates deep reinforcement learning (DRL) with traditional control techniques to fully leverage advanced decision-making and computational efficiency. Specifically, we develop a position controller for the leader UAV, combining the traditional Artificial Potential Field (APF) algorithm with the Deep Deterministic Policy Gradient (DDPG) algorithm. We design a potential-based action space and reward function for this controller. In addition, we have also designed a formation controller for the follower UAVs based on consensus theory, including the design of the control law, state switching conditions, and an obstacle and collision avoidance strategy based on Artificial Potential Field. This ensures that the formation safely reaches the target position while maintaining the formation. Finally, simulation experiments validate the effectiveness of the proposed method in enhancing formation efficiency and robustness, thereby offering a novel solution for multi-UAV missions in complex scenarios.