Multi-Intersection Signal Control Based on Asynchronous Reinforcement Learning

State-of-the-art theoretical models and new traffic signal control technologies are key guarantees for improving the management and safety performance of transportation systems, and multiagent reinforcement learning (MARL) methods have been widely applied in the field of signal control. Researchers in the transportation domain have effectively addressed the issues of poor convergence and suboptimal optimization encountered in RL for multi-intersection signal control scenarios by adopting the centralized training with decentralized execution (CTDE) approach. However, due to the heterogeneity among intersections, simply decomposing the global reward into a sum of intersection-level rewards is unreasonable, posing a challenge in balancing the interests of individual intersections and the entire road network. Additionally, the assumption that all intersections within the system make decisions synchronously is rather strong. Therefore, this paper proposes a distributed traffic model tailored for synchronous decision-making and, based on that, introduces an asynchronous decision-making traffic model according to decoupled intersection control. Simulation experiments show that the asynchronous decision-making method proposed in this paper not only improves the model convergence speed by at least 19% compared to the multiagent deep RL (MADRL) algorithm used for synchronous decision-making, but also improves the model by at least 10.5% in vehicle driving speed, maximum queue length, and average queue length within the decodable range (the traffic density is between 100 vehicles/km and 400 vehicles/km). In the same traffic scenario, the MADRL algorithm used for asynchronous decision-making has improved the average vehicle delay and average queue length by at least 55% compared to traditional arterial green wave control methods and adaptive control methods, and by at least 5% compared to SAC and A2C methods.