Adversarial Patch Steganography Enhancement through Localized Style Fusion

Since the concept of adversarial examples was introduced, various adversarial attack methods targeting deep learning models have raised a series of security issues. Among them, adversarial patches introduce specific patches into input samples to cause deep learning models to produce misleading results, posing significant security risks to current deep learning systems. However, current adversarial patch generation methods still have certain limitations in enhancing the concealment of adversarial patches, as they are prone to being detected by humans due to significant differences from the surrounding environment. In response to this problem, this paper proposes a method for enhancing the concealment of adversarial patches based on local style fusion. The method first searches for the vulnerable regions of the image based on multi-model weighted class activation mapping, and accurately locates the placement of adversarial patches, which improves the aggressiveness of adversarial patches. Then the style migration technique is utilized to compute the style matrix and content matrix of the target image and the adversarial patch. During the adversarial patch generation process, a comprehensive approach takes into account not only classification loss but also style loss, content loss, and boundary loss. By utilizing the cosine distance function, adjustments are made to both the style and content of the generated adversarial patch. This intricate adjustment ensures that the adversarial patch seamlessly blends its style and content with the local image it overlays, effectively integrating it into the surrounding environment. This harmonization guarantees consistency in color and style, ultimately minimizing the visibility of the adversarial patch to the human eye. The ultimate goal is to significantly enhance the concealment of the adversarial patch. In this paper, the generated patches are experimentally evaluated in terms of aggressiveness and covertness, respectively, and the experimental results show that this method can generate adversarial patches with both covertness and aggressiveness, which can realize the attack while being undetectable to human beings.