3D Path Planning for UAV with Improved Double Deep Q-Network

Liping Zhao; Yaofei Ma; Jie Zou

doi:10.1007/978-981-15-8458-9_40

3D Path Planning for UAV with Improved Double Deep Q-Network

Liping Zhao
, Yaofei Ma
, Jie Zou^*

^*Corresponding author for this work

School of Automation Science and Electrical Engineering

Beihang University
Science and Technology on Electro-Optic Control Laboratory of Luoyang

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Scopus citations

Abstract

Unmanned aerial vehicle (UAV) has been widely used in civil and military fields due to its advantages such as zero casualties, low cost and strong maneuverability. Path planning in 3D obstacle environment is one of the fundamental capabilities of UAV for mission performing. In this paper, we propose a 3D path planning algorithm to learn a target-driven end-to-end model based on an improved double deep Q-network (DQN), where a greedy exploration strategy is applied to accelerate learning. The model takes target and obstacle message as input, and moving command of UAV as output. It can realize path planning successfully for UAV in 3D complex environment. Besides, the experimental results show that improved double DQN has better convergence speed compared with DQN and double DQN.

Original language	English
Title of host publication	Proceedings of 2020 Chinese Intelligent Systems Conference - Volume II
Editors	Yingmin Jia, Weicun Zhang, Yongling Fu
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	374-383
Number of pages	10
ISBN (Print)	9789811584572
DOIs	https://doi.org/10.1007/978-981-15-8458-9_40
State	Published - 2021
Event	Chinese Intelligent Systems Conference, CISC 2020 - Shenzhen, China Duration: 24 Oct 2020 → 25 Oct 2020

Publication series

Name	Lecture Notes in Electrical Engineering
Volume	706 LNEE
ISSN (Print)	1876-1100
ISSN (Electronic)	1876-1119

Conference

Conference	Chinese Intelligent Systems Conference, CISC 2020
Country/Territory	China
City	Shenzhen
Period	24/10/20 → 25/10/20

Keywords

3D path planning
Greedy exploration strategy
Improved double DQN
Reinforcement learning
UAV

Access to Document

10.1007/978-981-15-8458-9_40

Cite this

Zhao, L., Ma, Y., & Zou, J. (2021). 3D Path Planning for UAV with Improved Double Deep Q-Network. In Y. Jia, W. Zhang, & Y. Fu (Eds.), Proceedings of 2020 Chinese Intelligent Systems Conference - Volume II (pp. 374-383). (Lecture Notes in Electrical Engineering; Vol. 706 LNEE). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-15-8458-9_40

@inproceedings{b6961431f5f1445f850e813554d1022f,

title = "3D Path Planning for UAV with Improved Double Deep Q-Network",

abstract = "Unmanned aerial vehicle (UAV) has been widely used in civil and military fields due to its advantages such as zero casualties, low cost and strong maneuverability. Path planning in 3D obstacle environment is one of the fundamental capabilities of UAV for mission performing. In this paper, we propose a 3D path planning algorithm to learn a target-driven end-to-end model based on an improved double deep Q-network (DQN), where a greedy exploration strategy is applied to accelerate learning. The model takes target and obstacle message as input, and moving command of UAV as output. It can realize path planning successfully for UAV in 3D complex environment. Besides, the experimental results show that improved double DQN has better convergence speed compared with DQN and double DQN.",

keywords = "3D path planning, Greedy exploration strategy, Improved double DQN, Reinforcement learning, UAV",

author = "Liping Zhao and Yaofei Ma and Jie Zou",

note = "Publisher Copyright: {\textcopyright} 2021, The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; Chinese Intelligent Systems Conference, CISC 2020 ; Conference date: 24-10-2020 Through 25-10-2020",

year = "2021",

doi = "10.1007/978-981-15-8458-9\_40",

language = "英语",

isbn = "9789811584572",

series = "Lecture Notes in Electrical Engineering",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "374--383",

editor = "Yingmin Jia and Weicun Zhang and Yongling Fu",

booktitle = "Proceedings of 2020 Chinese Intelligent Systems Conference - Volume II",

address = "德国",

}

Zhao, L, Ma, Y & Zou, J 2021, 3D Path Planning for UAV with Improved Double Deep Q-Network. in Y Jia, W Zhang & Y Fu (eds), Proceedings of 2020 Chinese Intelligent Systems Conference - Volume II. Lecture Notes in Electrical Engineering, vol. 706 LNEE, Springer Science and Business Media Deutschland GmbH, pp. 374-383, Chinese Intelligent Systems Conference, CISC 2020, Shenzhen, China, 24/10/20. https://doi.org/10.1007/978-981-15-8458-9_40

3D Path Planning for UAV with Improved Double Deep Q-Network. / Zhao, Liping; Ma, Yaofei; Zou, Jie.
Proceedings of 2020 Chinese Intelligent Systems Conference - Volume II. ed. / Yingmin Jia; Weicun Zhang; Yongling Fu. Springer Science and Business Media Deutschland GmbH, 2021. p. 374-383 (Lecture Notes in Electrical Engineering; Vol. 706 LNEE).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - 3D Path Planning for UAV with Improved Double Deep Q-Network

AU - Zhao, Liping

AU - Ma, Yaofei

AU - Zou, Jie

PY - 2021

Y1 - 2021

N2 - Unmanned aerial vehicle (UAV) has been widely used in civil and military fields due to its advantages such as zero casualties, low cost and strong maneuverability. Path planning in 3D obstacle environment is one of the fundamental capabilities of UAV for mission performing. In this paper, we propose a 3D path planning algorithm to learn a target-driven end-to-end model based on an improved double deep Q-network (DQN), where a greedy exploration strategy is applied to accelerate learning. The model takes target and obstacle message as input, and moving command of UAV as output. It can realize path planning successfully for UAV in 3D complex environment. Besides, the experimental results show that improved double DQN has better convergence speed compared with DQN and double DQN.

AB - Unmanned aerial vehicle (UAV) has been widely used in civil and military fields due to its advantages such as zero casualties, low cost and strong maneuverability. Path planning in 3D obstacle environment is one of the fundamental capabilities of UAV for mission performing. In this paper, we propose a 3D path planning algorithm to learn a target-driven end-to-end model based on an improved double deep Q-network (DQN), where a greedy exploration strategy is applied to accelerate learning. The model takes target and obstacle message as input, and moving command of UAV as output. It can realize path planning successfully for UAV in 3D complex environment. Besides, the experimental results show that improved double DQN has better convergence speed compared with DQN and double DQN.

KW - 3D path planning

KW - Greedy exploration strategy

KW - Improved double DQN

KW - Reinforcement learning

KW - UAV

UR - https://www.scopus.com/pages/publications/85092747980

U2 - 10.1007/978-981-15-8458-9_40

DO - 10.1007/978-981-15-8458-9_40

M3 - 会议稿件

AN - SCOPUS:85092747980

SN - 9789811584572

T3 - Lecture Notes in Electrical Engineering

SP - 374

EP - 383

BT - Proceedings of 2020 Chinese Intelligent Systems Conference - Volume II

A2 - Jia, Yingmin

A2 - Zhang, Weicun

A2 - Fu, Yongling

PB - Springer Science and Business Media Deutschland GmbH

T2 - Chinese Intelligent Systems Conference, CISC 2020

Y2 - 24 October 2020 through 25 October 2020

ER -

3D Path Planning for UAV with Improved Double Deep Q-Network

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this