A modified Q-learning algorithm for robot path planning in a digital twin assembly system

Product assembly is an important stage in complex product manufacturing. How to intelligently plan the assembly process based on dynamic product and environment information has become a pressing issue that needs to be addressed. For this reason, this research has constructed a digital twin assembly system, including virtual and real interactive feedback, data fusion analysis, and decision-making iterative optimization modules. In the virtual space, a modified Q-learning algorithm is proposed to solve the path planning problem in product assembly. The proposed algorithm speeds up the convergence speed by adding a dynamic reward function, optimizes the initial Q table by introducing knowledge and experience through the case-based reasoning algorithm, and prevents entry into the trapped area through the obstacle avoiding method. Finally, the six-joint robot UR10 is taken as an example to verify the performance of the algorithm in the three-dimensional pathfinding space. The experimental results show that the performance of the modified Q-learning algorithm is significantly better than the original Q-learning algorithm in both convergence efficiency and the optimization effect.