PointRL: Reinforcement Learning-Based Approach for Air-Ground Communications Using Multi-Dimensional Target Sensing Point Cloud

Unmanned aerial vehicles (UAVs) are increasingly used in smart city communications for air-ground communications due to their flexibility, low cost, and independence from ground conditions, enabling high data rates for future networks. This paper explores UAV-to-vehicle (U2V) mmWave integrated sensing and communication (ISAC), where vehicles are represented as rigid shapes in a 3D radar point cloud. Considering maximizing channel capacity with multi-user interference and radar performance, two adaptive optimization problems are proposed, incorporating vehicle-to-vehicle (V2V) communication for interference mitigation. The radar point cloud-driven reinforcement learning (PointRL) algorithm is designed to solve these problems. It includes a point cloud-based deep neural network (PDNN) for extracting action spaces from 3D radar data and a decision network that reduces network complexity through segmentation and connection. A linear weighted sliding window reward mechanism is also designed to enhance decision-making in dynamic environments. Simulation results show that the proposed PointRL outperforms benchmark methods.