QPCR: A Power Controlled Routing Protocol Based On Q-learning For Energy-constrained Underwater Wireless Sensor Networks

For energy-constrained Underwater Wireless Sensor Networks (UWSNs), it is challenging to design a lifetime-extended routing protocol due to high energy consumption, limited battery energy, and difficult sensor node replacement. A number of routing protocols have been proposed to address this issue. However, shortcomings still exist such as inefficient energy consumption and unbalanced workload among nodes, which make them unsuitable for energy-constrained UWSNs. To this end, we propose a power controlled routing protocol based on Q-learning (QPCR) with less energy consumption, more balanced workloads, and greater flexibility. Specifically, it adaptively switches between routing using different powers to extend the network lifetime or reduce the latency of urgent packets, guided by a Q-learning framework that jointly considers residual energy and packet urgency. In addition, to further extend the network lifetime, we design an energy efficiency-related reward in Q-learning to address premature energy exhaustion of hot spots near the sink. Simulation results demonstrate that QPCR outperforms QELAR, DBR, and PCDBR in terms of network lifetime, and QPCR reduces the latency of urgent packets through power control.