基于分层强化学习的天车调度优化方法

Cranes are key heavy-duty material handling equipment widely used in shops, warehouses, ports, and other industrial settings. The scheduling of cranes significantly affects transportation efficiency and the achievement of production goals. To address the crane scheduling problem with time windows (CSP-TW), a mixed-integer linear programming model based on spatio-temporal discretization is developed. Based on the characteristics of the model, a hierarchical reinforcement learning (HRL) decision-making framework is designed. The high-level decision network assigns transportation tasks to appropriate cranes, while the low-level network plans paths for each crane to complete its assigned task. During the learning process, action tabu rules are introduced to avoid ineffective actions and guide the decision networks toward the dominant policy space. Subsequently, external experience pooling and the dueling double deep Q-network strategy are adopted to train the decision networks. Tests were executed based on the logistics simulation platform of a steel plant from a certain company. Ablation experiments show that the introduction of action tabu rules improves learning efficiency.Training comparisons indicate that HRL achieves better convergence than the end-to-end framework. Comparative experiments demonstrate that HRL outperforms several methods, including multi-rule combinations, meta-heuristic algorithms, end-to-end and deep Q-network, while satisfying second-level response-time requirements for applications.