A Hierarchical Game-Theoretic Based Deep Reinforcement Learning Approach for Aircraft Conflict Resolution

Rapid growth in air traffic flow results in more complex and congested airspace, increasing the likelihood of conflicts among aircraft during operations. Conflict resolution in such airspace is intractable, especially under an approach convergence flight scenario, where multiple approach routes come together, creating numerous potential conflict points. A critical gap remains, as most conflict resolution approaches neglect common aircraft priority constraints during these convergence operations. To address these challenges, we propose a hierarchical game-theoretic deep reinforcement learning (HGT-DRL) conflict resolution framework. This framework classifies aircraft into Leader Aircraft (LA) and Follower Aircraft (FAs), and models the inter-aircraft interaction structure using multiple layers. The relationship between the LA and FAs is formulated as a Stackelberg game, while the coordination among FAs is represented through a Nash equilibrium. To realize efficient calculation, the combined game model is transformed into a multi-agent Markov Decision Process, which is solved using Proximal Policy Optimization for the LA and Multi-Agent PPO for the FAs. By integrating structured game theory with deep reinforcement learning, the proposed approach supports scalable, priority-aware, and real-time conflict resolution. We implement the HGT-DRL method, train it under an approach convergence flight scenario, and present the training results. A comparative experiment against a state-of-the-art baseline is conducted to evaluate performance. The HGT-DRL method demonstrates strong conflict-resolution capability and exhibits a structured, tiered decision-making process during convergence flight.