MULTI-OBJECTIVE AERODYNAMIC OPTIMIZATION OF 2D HIGH-LIFT DEVICE BASED ON DISTRIBUTED DEEP REINFORCEMENT LEARNING

The design of high-lift device (HLD) is of great significance for large aircraft, which can effectively improve the take-off, landing performance and safety. The aerodynamic design of HLD is a multi-objective optimization problem which consumes a lot of computing resources. In our previous work, we proposed a method based on deep reinforcement learning (DRL) and transfer learning (TL), which transfers the experience of two-dimensional (2D) optimization to three-dimensional (3D) optimization. However, for the multi-objective optimization problem, this method still merges several objectives into a single objective through linear weighted superposition, which is mainly due to the fact that the loss function of neural network and the reward function of Reinforcement Learning (RL) must be scalar. In orderto obtain the ability to find the complete Pareto solution similar to heuristic algorithms, we further improve the algorithm by combining the idea of distributed reinforcement learning which is still based on the Deep Deterministic Policy Gradient. The new algorithm changes the original single actor into multiple actors, and each actor trials and errors in the environment. A Critic is used to score all the actors' actions and give the correct value gradient. Three reward functions are compared and the function based on Hypervolume is better. The trailing-edge high lift device of a commercial aircraft is optimized and the agent can find a set of relatively complete Pareto solutions.