Hierarchical Extraction of Team Tactical Strategies and Multi-Level Interpretability Analysis in Multi-Agent Reinforcement Learning

The lack of interpretability in team cooperative behaviors within multi-agent reinforcement learning (MARL) systems hinders the validation of tactical knowledge and its practical applications. This paper targets the domain of multi-aircraft intelligent games and proposes a trajectory-analysis-based framework for hierarchical extraction of team tactical strategies and multi-level interpretability analysis, enabling systematic mining of team tactical knowledge. The framework addresses spatial and directional variations in multi-aircraft trajectories by designing situation normalization preprocessing and autoencoder-based latent space representation; it employs HDBSCAN combined with genetic algorithm for intra-trajectory segmentation (HD-GA) and HDBSCAN-spectral clustering integration (HD-SP) to extract elementary team tactics (ETT) and composite team tactics (CTT); furthermore, it utilizes Kernel SHAP to perform feature importance analysis on key decision points, forming a complete interpretability closed loop from macro to micro levels. Experiments on a dataset comprising 200 trajectories covering four typical team tactics validate the framework, where HD-SP achieves a CTT recognition accuracy of 0.95, and HD-GA effectively segments ETT while extracting tactical phases consistent with strategic logic. This work significantly enhances the interpretability and credibility of MARL systems in multi-aircraft intelligent games, providing an efficient tool for transparentizing complex multi-agent cooperative decision-making, with potential for extension to other scenarios.