An Adaptive Online Parameter Control Algorithm for Particle Swarm Optimization Based on Reinforcement Learning

Parameter control is critical to the performance of any evolutionary algorithm (EA). In this paper, we propose a Q-Learning-based Particle Swarm Optimization (QLPSO) algorithm, which uses the Reinforcement Learning (RL) to train the parameters in Particle Swarm Optimization (PSO) algorithm. The core of the QLPSO algorithm is a three-dimensional Q table which consists of a state plane and an action axis. The state plane includes the state of the particles in both of the decision space and the objective space. The action axis controls the exploration and exploitation of particles by setting different parameters. The Q table can help particles to select actions according to their states. Besides, the Q table should be updated by reward function which is designed according to the performance change of particles and the number of iterations. The main difference between the QLPSO algorithms for single-objective and multi-objective optimization lies in the evaluation of the solution performance. In single-objective optimization, we only compare the fitness values of solutions, while in multi-objective optimization, we need to discuss the dominant relationship between solutions with the help of Pareto front. The performance of QLPSO is tested based on 6 single-objective and 5 multi-objective benchmark functions. The experiment results reveal the competitive performance of QLPSO compared with other algorithms.