Predictive Risk-Aware MARL-Based Cooperative Driving Strategy for CAVs in Highly Interactive Driving Environments

In highly interactive driving environments, the collision accidents and severe congestion often occur due to the multi-modal driving behaviors and dynamic interactions of surrounding vehicles. Moreover, the balance between individual and overall traffic is hard to maintain as well. Such challenging scenarios have become a major obstacle for its real-world application. Owing to the significant advancements of the IoT and AI, multi agent method and the awareness of driving risk have become promising way to enhance the safety problems. To improve the safety and efficiency in highly dynamic and interactive environment, a predictive risk-aware cooperative driving policy is proposed based on multi-agent reinforcement learning (MARL) framework. Firstly, considering multiple future behaviors and dynamic evolution of surrounding vehicles, a predictive risk field is established based on elliptical collision boundary. And then, the predictive risk is introduced in our cooperative driving policy to enhance the safety in highly interactive environment. Furthermore, efficiency of individual vehicle and overall traffic flow are balanced when we design the reward function. Subsequently, to learn the optimal cooperative driving policy, an advantage actor-critic algorithm is also designed, in which the predictive risk is integrated into the centralized critic, facilitating comprehension of global traffic situation and coordinate all vehicles to ensure safety. Moreover, the risk-aware critic guides the policy updating of decentralized actor for each controlled vehicle. In this way, our approach could improve the comprehension of interactive and dynamic traffic environment. The results demonstrate that the introduction of predictive risk and multi-agent systems can make a trade off between individual and overall traffic efficiency within safety constraints, even in challenging traffic situations with high-density flows.