All-aspect attack guidance law for agile missiles based on deep reinforcement learning

This paper presents an all-aspect attack guidance law for agile missiles based on deep reinforcement learning (DRL), which can effectively cope with the aerodynamic uncertainty and strong nonlinearity in the high angle-of-attack (AOA) flight phase. First, to make the training environment more authentic, the full flight envelope of the missile is modeled and highly accurate aerodynamic data is obtained through Computational Fluid Dynamics (CFD) technique. Subsequently, the DRL algorithm is applied to generate an AOA guidance law for the agile turn phase. A hierarchical scheme that consists of a meta-controller for real-time decision making according to combat scenario and a sub-controller for generating guidance command is designed, which enables the guidance law to cover the whole process of the engagement and ensures the convergence of the training in the agile turn phase. Considering the current limitations of missile maneuverability, two agile turn guidance laws are developed to accommodate both limited and unlimited AOA scenarios. Moreover, the proposed guidance law has excellent generalization capability and ensures the implementation of static training and dynamic execution, which means that the missile can adapt to the realistic combat scenarios that have not been encountered during the training. Simulation results indicate that the DRL-based guidance law is nearly optimal and robust to disturbances. In addition, the proposed guidance law enables the missile to track time-varying desired turn angles to lock the maneuvering target in the rear hemisphere during the agile turn phase, providing advantageous initial conditions for the terminal guidance. Furthermore, the computational efficiency is high enough to satisfy the requirement on onboard application.