An End-to-End Deep Reinforcement Learning Method for UAV Autonomous Motion Planning

UAVs rely on mapping the surroundings to gather real-time environmental information. The mapping outcome stands as the key prerequisite for further motion planning. Plenty of work has been done with sophisticated mapping algorithms. However, during UAV navigation, these algorithms are expected to be updated continuously, which consumes a significant amount of memory and computational resources in a large area. To address this limitation, in this paper, we propose an end-to-end method for UAV autonomous motion planning via Reinforcement Learning (RL). In particular, a deep RL network is built as the brain of the intelligent agent, which takes the depth image of the UAV visual feedback as input, and outputs the continuous action as a control decision. A convolutional neural network is employed to process the depth image. In order to implement and validate the proposed method, a high-fidelity 3D simulation environment is established in AirSim, which generates the real-time flight status and depth images during the UAV flight. As a result, the flight simulation demonstrates the effectiveness and efficiency of the RL-based motion planning algorithm in a complex environment. Importantly, the agent trained by the proposed IDDPG could get closer to the destination than that trained by DDPG by about 17 meters on average. Last, the computation time for each step is significantly reduced to 5 ms compared to the classical approach.