Deep Reinforcement Learning-Driven Parameter Tuning for Adaptive Control Systems in Hypersonic Flight Vehicle

Hypersonic flight vehicle faces critical challenges of control from highly nonlinear and time-varying uncertainties, which impose stringent requirements for real-time parameter adaptation under safety constraints. This paper proposes a reinforcement learning-based adaptive tracking control algorithm to address these issues. The crucial contributions of our design, as opposed to the state-of-the-art approaches, lie in three aspects: 1) a hybrid design of model-based control and reinforcement learning to alleviate the safety, stability and generalization issues of learning-based methods specifically for the demanding hypersonic flight environment; 2) the establishment of a reinforcement learning-based optimization framework that dynamically adjusts control parameters in a real-time optimal fashion to improve the tracking performance under dynamic uncertainties and flight regime transitions, which is substantially different from most conventional methods with constant parameters; 3) the theoretical analysis of both the closed-loop stability of the adaptive control and the convergence performance of the learning algorithm, which distinguishes our design from most existing reinforcement learning-based methods that have no stability or convergence guarantee and is particularly critical for safety-critical hypersonic flight vehicle applications. Numerical simulations show that the proposed method achieves a reduction in the integral of tracking error of 8.31% under model perturbations and 34.3% under changing reference trajectories, compared to the baseline method, while maintaining comparable control energy consumption.