Parametrized Graph Convolutional Multi-Agent Reinforcement Learning with Hybrid Action Spaces in Dynamic Topologies

Multi-agent swarm collaboration, inspired by the collective behaviors of biological swarms in nature, has wide applications in dynamic open environments. However, hybrid action spaces in multi-agent reinforcement learning (MARL) present a critical challenge: the inherent coupling between discrete and continuous actions severely undermines policy stability and convergence, especially under dynamic topologies. Existing methods fail to decouple this coupling, leading to suboptimal policies and unstable training. This paper addresses the core problem of action coupling under dynamic topologies, proposing a Parametrized Graph Convolution Reinforcement Learning (P-DGN) method. Operating within the actor–critic framework, P-DGN decouples the optimization pathways for hybrid actions, with a biomimetic observation design inspired by starling flock behaviors: each agent only observes the states of its seven nearest neighbors to achieve efficient local interaction and global collaboration. Its actor network uses multi-head attention to build dynamic relation kernels, develops temporal relation regularization (TRR) to improve policy consistency across time steps, and generates continuous actions with a Gaussian policy. Meanwhile, P-DGN’s critic network, based on deep Q-network (DQN), evaluates Q-values for discrete actions to guide optimal choices. We evaluate P-DGN in two different multi-agent cooperative environments. Experimental results show that compared with parametrized deep Q-network (P-DQN) and DQN baseline, the proposed method has faster convergence speed and stronger training stability. Moreover, with dense rewards, P-DGN agents learn emergent tactics like encirclement. Overall, P-DGN offers a new approach for optimizing hybrid action spaces in multi-agent systems within open, dynamic environments, balancing theoretical generality with practical utility, and its biomimetic design provides a biologically plausible framework for multi-agent swarm collaboration.