A multi-agent deep reinforcement learning driven adaptive construction and search algorithm for time-varying agile Earth observation satellite scheduling

The difficulty of complex scheduling problems is significantly affected by the increase in task scale. A multi-agent deep reinforcement learning-guided adaptive construction and search (MADRL-ACS) algorithm is proposed to solve agile Earth observation satellite scheduling problems (AEOSSP). The MADRL-ACS algorithm, a combination of ensemble learning and data analytics strategies, was employed to address the challenges of complex, time-varying task constraints, multi-objective conflict optimization, and insufficient computational efficiency in large-scale AEOSSP. The multi-core computational resources are used in the ensemble learning approach by integrating classical heuristic algorithms, namely hill climbing (HC), Tabu search (TS), and simulated annealing (SA), with deep reinforcement learning (DRL). A multi-agent framework is adopted in the ensemble learning approach, where each agent employs a dueling deep Q-network (DQN) architecture to select local search strategies based on real-time state information and to execute them intelligently within each parallel thread. Density-based spatial clustering of applications with noise (DBSCAN) is used in the data analytics strategy to guide frequent pattern mining, whereby the elite solution set is periodically analyzed for density and dynamically partitioned into clusters. Frequent patterns are mined from representative solutions to construct new candidates, thereby enhancing global search capability and effectively preventing premature convergence. Experimental results demonstrate that MADRL-ACS outperforms the other compared algorithms in terms of both solution quality and computational efficiency, and exhibits strong robustness and practical applicability in large-scale, complex scheduling scenarios.