Hierarchical deep reinforcement learning based multi-agent game control for energy consumption and traffic efficiency improving of autonomous vehicles

To achieve highly autonomous driving while ensuring eco-driving, this paper proposes a Hierarchical Multi-Agent Deep Reinforcement Learning framework to optimize energy consumption and traffic efficiency for autonomous vehicles. In this framework, driving, braking, traffic efficiency, and energy management are modeled as independent agents within a game-theoretic framework. Distinct reward functions are designed to establish cooperative and competitive relationships among the agents based on training objectives. Initially, path planning and obstacle detection are implemented in the CARLA simulation environment, where deep learning algorithms enhance trajectory tracking and real-time decision-making. Incorporating complex urban environmental factors such as traffic signals and vehicle interactions, a multi-objective hierarchical optimization strategy is proposed to balance energy consumption, traffic efficiency, and driving safety. For energy management, an expert-knowledge-guided multi-agent learning mechanism is introduced to reduce the search space and accelerate convergence, achieving improved energy efficiency and decision-making stability. Simulation results demonstrate that, compared to the traditional Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3) method, the proposed Expert-MATD3 method reduces energy consumption by 10%, shortens travel time by approximately 20.37%, and exhibits the slowest state of charge(SOC) decline, further demonstrating its superior energy management efficiency while maintaining a high level of driving safety. Moreover, the method exhibits strong generalization capability and real-time performance, providing a promising approach for sustainable and efficient autonomous driving.