Multiscale-Graph-Enhanced Reinforcement Learning for Conflict Resolution in Dense UAV Networks

Effective conflict resolution (CR) is crucial to ensure the safety of uncrewed aerial vehicles (UAVs) in increasingly congested low-altitude airspace. However, the inefficiency in representing large-scale UAV information hinders the performance of existing learning-based methods. To overcome this challenge, we propose an enhanced graph-based reinforcement learning approach that models multiagent interaction relationships. Specifically, a novel multiscale graph reinforcement learning (MS-GRL) approach is utilized to learn UAV avoidance strategies in a dense environment. MS-GRL utilizes the time-evolving intensity of local conflicts to evaluate the global attention weights for UAVs in conflict while aggregating UAV observation data through graph embedding and requisite feature refinement. In addition, to adaptively limit the safety region of the action space while minimizing deviations from the original trajectory, a safety-constrained maneuver strategy is proposed. The experimental results demonstrate that MS-GRL outperforms state-of-the-art GRL methods when there are up to 100 UAVs and multiple obstacles.