Heterogeneous formation control of multiple rotorcrafts with unknown dynamics by reinforcement learning

In this paper, a distributed model-free solution based on reinforcement learning is proposed for the leader–follower formation control problem of heterogeneous multi-agent systems. The multi-agent system consists of multiple rotorcrafts involving a virtual leader and multiple followers, where the dynamics of leaders and followers is unknown. The formation control problem is firstly formulated as an optimal output regulation problem. A discounted performance function is then introduced to guarantee that the tracking error asymptotically converges to zero, and an online off-policy reinforcement learning algorithm is proposed to solve the optimal output problem online using the data generated along the trajectories of the agents. A simulation example is provided to validate the effectiveness of the proposed control method.