Multi-Scale High-Frequency Vision Transformer for Face Forgery Detection

In recent years, as more and more mature face forgery methods have been disclosed, the forged face has caused a lot of negative effects in the society, so people hope to develop effective face forgery detection methods to solve this problem. Although a lot of recent work has achieved good performance, there is still room for improvement in the forgery face which is forged by unknown means. Based on the recent observation and analysis, there are two promising ways that may improve the performance of face forgery detection methods, i.e., effectively exploiting the high-frequency features and reducing the interference of identity information. In this paper, we consider both of these aspects and the powerful feature representation capability of Vision Transformer (ViT), and propose a Multi-scale High-frequency Vision Transformer (MH-ViT) method that can effectively use the multi-scale high-frequency features of face images. In order to reduce the interference of identity information to the face forgery detection method as much as possible, this paper randomly partitions a face image into blocks to weaken the identity information, and designs the Patch SoftMax (PSM) loss function. Moreover, to supplement the PSM loss function, we also add a metric learning module and design the Single Center Cosine Similarity (SCCS) loss function with a large margin to expand the distance between real and fake samples. Extensive experiments on both in-domain and cross-domain face forgery detection datasets demonstrate the effectiveness of our proposed MH-ViT method.