Agents Trainer: Automatically Training Multi-Agent Reinforcement Learning Models for Drone Swarm Using Language Model-Based Agents

Large language models (LLMs) excel at high-level reasoning and task decomposition but remain insufficiently reliable for fine-grained, low-level control, where reinforcement learning (RL) is more suitable. However, effective RL still relies heavily on expert-intensive environment configuration, reward design, and hyperparameter tuning–challenges that are amplified in multi-agent scenarios. To this end, we propose Agents Trainer, an LLM-assisted framework that automates the end-to-end multi-agent RL (MARL) pipeline with minimal expert intervention. The system decomposes MARL responsibilities across specialized agents and instantiates a generate-reflect-evolve iteration loop to produce executable, interpretable, and progressively densified reward signals aligned with task goals and safety constraints, while preserving diversity to avoid mode collapse. Experiments across two simulators of differing fidelity and a diverse set of tasks show that Agents Trainer outperforms classical MARL methods and automated reward-design baselines in success rate, episode return, robustness, and sample efficiency, and shifts the safety–efficiency Pareto frontier toward fewer collisions and lower energy consumption. Ablation studies demonstrate that removing any component degrades reliability or stability. A multi-granularity Sim2Real pipeline further validates that our method achieves lower trajectory error and jerk on physical quadrotors, enabling smoother and more precise transfer. These gains are reproducible across different underlying LLM backends, indicating that the performance improvements stem primarily from the framework design rather than any specific model choice. Note to Practitioners—This paper presents a practical approach to automating MARL in real-world robotics and automation systems by leveraging large language models (LLMs) to reduce expert effort in reward design and training protocols—tasks that are traditionally time-consuming and expertise-dependent. The implementation requires only minimal initial setup (e.g., specifying a high-level objective such as “achieve collision-free drone formation”) and is compatible with existing RL environments and simulators (e.g., OpenAI Gym, AirSim) through API-level integration. Currently, a single workstation can comfortably handle swarms of roughly 20–30 entities; larger teams typically require distributed or cloud execution. Caching rollouts and intermediate metrics can substantially reduce the overhead on the Eval and Evolution agents. The framework enables engineers to rapidly prototype MARL solutions for dynamic environments while maintaining alignment with operational goals via intuitive feedback. We recommend keeping reward functions modular and separating task rewards from safety penalties in both logging systems and dashboards. Future extensions will support ROS-based deployment to physical robots.