Vision-based Navigation of UAV with Continuous Action Space Using Deep Reinforcement Learning

Autonomous navigation of unmanned aerial vehicles (UAV) in an unknown environment is a challenging task that attracts many researchers. There exist many solutions to the problem, one promising method is deep reinforcement learning. In this paper, we investigate a deterministic policy based actor-critic learning framework on vision-based navigation of an autonomous UAV within a simulated environment. In particular, navigation with high-level input and continuous output is considered. In simulation, the observation of UAV is a depth image from front camera while the target information is a vector, to combine them together, we calculate the track angle between UAV and target, and encode it into a depth image together observation, making up to state representation. In the framework of our algorithm, actor network adopts convolutional layer to deal with high-level input, while critic network employs merge layer to balance state information and action information. The result of the experiment supports the idea of full control of an autonomous UAV through deep reinforcement learning as we solve the task successfully. Besides, comparison with other method was conducted to further explore the advantage of the method.