TSBA: A two-stage poison-only backdoor attack on visual object tracking

Training high-performance Visual Object Tracking (VOT) models often relies on third-party resources, making these models vulnerable to backdoor attacks. In such attacks, attackers can implant backdoors by poisoning the training dataset and manipulating the model training process. Existing backdoor attack methods for VOT assume that the attacker has complete control over the model training process, or the designed attacks are untargeted, which limits the practicality and effectiveness of these methods. To address this issue, we propose a Two-Stage Poison-Only Backdoor Attack (TSBA). Specifically, in a poison-only scenario, TSBA employs a two-stage poisoning strategy to attach triggers to both the object region and the selected background region in video frames, while using contrastive loss and total variation loss to optimize the triggers, enhancing the effectiveness and stealthiness of the attack. Extensive experiments under various settings show that our backdoor attack significantly degrades the performance of trackers based on Siamese networks, Transformer, and temporal information, outperforming existing attack methods. Moreover, we validate the robustness of our attack against several potential backdoor defenses.