An Enhanced Adviser-Actor-Critic Framework for High-Precision Reinforcement Learning Control

High-precision control of complex systems faces challenges due to nonlinear dynamics and limitations of traditional methods. Our previous Adviser-Actor-Critic (AAC) framework integrates PID control with reinforcement learning (RL), using PID as an adviser to guide the agent toward the desired position via”fake goals” to minimize steady-state error. In this paper, we introduce three refinements to AAC: advanced PID design, Bayesian optimization for parameter tuning, and hybrid feedforward-feedback control, to address issues like integral windup, manual parameters tuning, and latency. Ablation experiments demonstrate that AAC-T outperforms standard AAC in steady-state accuracy and response speed, validating its effectiveness for high-precision control tasks in robotics and aerospace applications.