Reinforcement Learning Control of A Novel Magnetic Actuated Flexible-joint Robotic Camera System for Single Incision Laparoscopic Surgery

This paper describes the control of a novel Magnetic Actuated Flexible-joint Robotic Surgical (MAFRS) camera system with four degrees of freedom (4-DOF) for single incision laparoscopic surgery. Based on the idea of motion decoupling, we designed a novel MAFRS system which is consists of an external driving device and a motor-free insertable wireless robotic device with a hollow flexible joint. Due to the problems of abdominal wall obstruction and variability in abdominal wall thickness during the actual application of the MAFRS system, as well as the existence of multiple permanent magnets and magnetically conductive media, high-precision position and attitude control of the insertable device without onboard motors has always been a challenge. We use the external driving device to generate a magnetic field to control the position and attitude of the internal robotic device. Aiming at the automatic precise tilt motion control of the novel MAFRS camera system, we have developed a closed-loop control scheme using the Deep Deterministic Policy Gradient (DDPG) algorithm. By referring to the damping characteristics of human muscles, a virtual-muscle method is proposed to eliminate the chattering problem of the MAFRS camera at specific angles. The experimental investigations indicate that the internal robotic device can be effectively controlled under different abdominal wall thicknesses. The tilt motion control accuracy is within 0.5^◦, and it has good adaptability and anti-interference performance.