Unmanned Aerial Vehicle Autonomous Visual Landing through Visual Attention-Based Deep Reinforcement Learning

Unmanned aerial vehicle (UAV) autonomous landing is still an open and challenging issue. State-of-art work focused on the use of meticulously designed hand-crafted geometric features and complex sensor-data fusion to identify fiducial marker and guide a UAV. To address these issues, this study proposed a novel end-to-end control method based on deep reinforcement learning (DRL) that only requires low-resolution images of the environment ahead. A convolutional neural network (CNN) function approximator combined with visual attention blocks was adopted for direct frame-to-action prediction. The input frames were acquired from a low-cost monocular camera integrated with the UAV, without any other sensors. The combination of a dueling deep Q-network (DQN) with diverse visual attention modules was implemented and compared with the original dueling DQN. The simulation results showed that the UAV could autonomously land on a marker in a high-fidelity virtual simulation environment with rich scene transformation, regardless of initial relative positions. Moreover, the test of the proposed visual landing algorithm in multiple actual landing scenarios proved that the algorithm can accurately extract and localize the landing marker by using real images.