Hierarchical Soft Actor-Critic and Model Predictive Control Framework for Robotic Manipulator Grasping

Active visual tracking plays a critical role in vision-guided robotic manipulation, where the robot needs to continuously maintain target visibility while generating smooth and dynamically feasible motions. Although deep reinforcement learning (DRL) methods such as soft actor-critic (SAC) can learn high-dimensional visuomotor representations, end-to-end DRL policies often suffer from stochastic actions, non-smooth motion profiles, and the absence of explicit physical and visibility constraints, limiting their reliability on real robotic systems. To address these challenges, a hierarchical SAC and model predictive control framework is proposed for visual-guided robotic manipulation in this paper. Specifically, the high-level SAC module functions as a motion planner that outputs task-space reference velocities based on visual and proprioceptive observations, bridging the gap between raw perception and low-level control. The low-level nonlinear model predictive control module optimizes joint torques by explicitly incorporating robot dynamics, joint and torque limits, singularity avoidance, and camera field-of-view (FOV) visibility constraints. It should be noted that this hierarchical decoupling enables the system to leverage strong perceptual capability of SAC while ensuring dynamically consistent and constraint-compliant execution. Experiments are conducted on 6-DOF UR5 manipulator in the MuJoCo physics simulator to validate the effectiveness and robustness for active visual tracking tasks The results demonstrates that the proposed SAC and model predictive control framework significantly improves pixel tracking accuracy, enforces near-zero field of view violation, and produces smoother control trajectories compared with pure SAC and classical image-based visual servoing (IBVS) baselines.