Two-Critic Deep Reinforcement Learning for Inverter-Based Volt-Var Control in Active Distribution Networks

A two-critic deep reinforcement learning (TC-DRL) approach for inverter-based volt-var control (IB-VVC) in active distribution networks is proposed in this paper. Considering two objectives of VVC, minimizing power loss and eliminating voltage violations, have different mathematical properties, we utilize two critics to approximate two objectives separately, which reduces the learning difficulties of each critic. The TC-DRL approach cooperates well with many actor-critic DRL algorithms for the centralized IB-VVC problems, and two centralized DRL algorithms were designed as examples. For decentralized IB-VVC, we extend the approach to a multi-agent TC-DRL approach and further simplify the multi-agent DRL approach with all agents sharing the same centralized two-critic. Extensive simulation experiments show that the proposed two centralized TC-DRL algorithms require fewer iteration times and return better results than the recent DRL algorithms, and the multi-agent TC-DRL algorithms work well for decentralized IB-VVC problems with different limited real-time measurement conditions.