Role-Policy Enhanced Collaborative Task Learning in Multiagent Systems

Current mainstream multiagent reinforcement learning (MARL) algorithms primarily focus on acquiring the global maximum reward throughout the entire training process, from the initial to the final stage. Whereas directly pursuing the global maximum return tends to be inefficient, particularly in environments with sparse rewards or the large-scale multiagent system. To address these challenges, previous algorithms have been developed to maintain individual policies to guide global training. Nevertheless, these approaches generally neglect either efficiency or the potential for local collaboration at the early stage of training. In this article, we propose the role-policy enhanced global policy (RPEGP) algorithm, which integrates the concept of distinct roles within the actor–critic-based MARL framework. RPEGP simultaneously considers both collaborative behaviors among agents and efficient global policy training. Specifically, RPEGP exploits the similarities among agents to assign distinct roles, training role-policies and the global policy concurrently. Through the initialization and enhancement of the role-policies, the global policy is trained more efficiently and effectively. Empirical experiments are conducted in well-known cooperative multiagent environments, including StarCraft II micromanagement (SMAC) and multiagent particle environment (MPE). Experimental results demonstrate that RPEGP outperforms baseline algorithms across various evaluation metrics and training efficiency, confirming its ability to address complex cooperative tasks generically and efficiently.