Mode-free altitude control for airship based on Q-learning and CMAC network

A mode-free airship altitude control strategy is proposed based on Q-learning algorithm and CMAC (Cerebellar Model Articulation Controller) network. First, we designed an adaptive method to establish MDP (Markov Decision Process) model of airship altitude control on the foundation of analyzing actual motions. Then, control strategy is learned online via Q-learning algorithm, and CMAC network is used to generalize the value function of action. Finally, typical control tasks are carried out using a simulated low altitude airship. Control strategy could be worked out without access to airship accurate dynamic model. Results show that the airship's MDP model parameters using the adaptive method are better matched with each other than using simple equally divided method, and CMAC networks could accelerate converge of algorithm. Control strategy's effectiveness and stability is demonstrated by comparing with a PID controller.