Accelerating wargaming reinforcement learning by dynamic multi-demonstrator ensemble

Deep Reinforcement Learning (DRL) has become a promising technique to deal with tough wargaming decision-making problems. However, DRL suffers an inherent problem of low learning efficiency and it often requires massive cost of training steps, which may be alleviated with expert demonstrations in wargaming domains. Most learning methods with demonstrations generally treat the demonstration data from different expert demonstrators without distinction. Besides, a more appropriate and effective mechanism is highly needed to control sampling balance of expert-generated demonstration samples and agent-generated interaction ones. To tackle the two issues, this work proposes an improved approach to leverage expert demonstrations to further accelerate DRL. It innovatively extracts inherent diversity in multiple demonstrators by pre-training agents individually from multiple demonstration sources, thereby producing a strong and initial ensemble model. In addition, a novel technique to evaluate the learning importance of each demonstrator is designed to dynamically tune sampling ratios of learning data in a more adaptive and effective manner. Through the evaluation on several classic game tasks and a typical wargaming scenario, our method shows superior performance over several state-of-the-art methods and significantly raises DRL's efficiency for typical wargaming decision-making applications.