Safe reinforcement learning for aerospace control via model-relaxed lyapunov stability

Ensuring stability and safety in reinforcement learning (RL) for aerospace control presents a critical challenge, as conventional model-free RL often lacks guarantees while model-based methods face computational burdens or rely on precise models. To address this, this paper proposes SAC–CLF, an integrated framework combining Control Lyapunov Functions (CLFs) with the Soft Actor-Critic (SAC) algorithm. The core of SAC–CLF integrates safety through three key mechanisms: first, a principled CLF design methodology automatically synthesizes an LQR-based CLF, ensuring both stability and local optimality; second, a Recursive Gaussian Process State-Space Model (RGPSSM) is integrated for real-time, uncertainty-aware learning of unmodeled dynamics, adaptively adjusting the CLF’s safety margin based on quantified epistemic uncertainty; and third, a safety-prioritised control input smoothing technique embeds a penalty for rapid control changes directly into the Quadratic Program (QP) objective. This comprehensive framework synergistically balances SAC’s exploratory efficiency with robust stability, adaptive performance, and the generation of high-quality, safe control actions. Experimental validation demonstrates SAC–CLF’s superior robustness, safety, and faster convergence compared to conventional CLF-based control and baseline RL methods, offering a promising direction for deploying RL in safety–critical aerospace domains.