Peg-in-hole assembly method based on visual reinforcement learning and tactile pose estimation

When robots replicate human actions in peg-in-hole assembly tasks, such as USB Type-A insertion and removal, the complexity of the process and frequent obstructions from the inner walls make it difficult for robots to handle collisions or avoid jamming. These difficulties contribute to a low success rate in assembly. This paper proposes a vision-guided reinforcement learning pre-assembly combined with tactile feedback-based pose estimation adjustment method for peg-in-hole assembly, achieving significant improvement in success rates for complex assembly tasks. First, during the pretraining process of reinforcement learning, high-reward sample data is collected, and a behaviour cloning (BC) algorithm is constructed based on sample data structure. The network is pretrained as a policy regression layer. Under sparse reward conditions, outputs of the twin delayed deep deterministic policy gradient (TD3) network and the BC network are combined to improve training stability and accelerate convergence, enhancing the efficiency of vision-based assembly. Then, to address the instability caused by collisions with the inner and outer walls of the hole when vision-based assembly remains incomplete, an in-hand pose estimation algorithm based on the Gelsight visuotactile sensor is integrated. This algorithm facilitates real-time adjustments to the position of the robot's end-effector, improving the likelihood of successful peg-in-hole assembly. Finally, to validate the effectiveness of the proposed method, experiments were conducted using the V-REP simulation platform and the real Franka robot platform. In the experiments, success rates of 90-93% and 80-85%, respectively, were achieved.