跳到主要导航 跳到搜索 跳到主要内容

Reliable and Balanced Transfer Learning for Generalized Multimodal Face Anti-Spoofing

  • Xun Lin
  • , Ajian Liu
  • , Zitong Yu*
  • , Rizhao Cai
  • , Shuai Wang
  • , Yi Yu
  • , Jun Wan
  • , Zhen Lei
  • , Xiaochun Cao
  • , Alex Kot
  • *此作品的通讯作者
  • Beihang University
  • CAS - Institute of Automation
  • University of Chinese Academy of Sciences
  • Great Bay University
  • Shenzhen University
  • Dongguan Key Laboratory for Intelligence and Information Technology
  • Nanyang Technological University
  • Macau University of Science and Technology
  • Chinese Academy of Sciences
  • Sun Yat-Sen University

科研成果: 期刊稿件文章同行评审

摘要

Face Anti-Spoofing (FAS) is essential for securing face recognition systems against presentation attacks. Recent advances in sensor technology and multimodal learning have enabled the development of multimodal FAS systems. However, existing methods often struggle to generalize to unseen attacks and diverse environments due to two key challenges: (1) Modality unreliability, where sensors such as depth and infrared suffer from severe domain shifts, impairing the reliability of cross-modal fusion; and (2) Modality imbalance, where over-reliance on a dominant modality weakens the model’s robustness against attacks that affect other modalities. To overcome these issues, we propose MMDG++, a multimodal domain-generalized FAS framework built upon the vision-language model CLIP. In MMDG++, we design the Uncertainty-Guided Cross-Adapter++ (U-Adapter++) to filter out unreliable regions within each modality, enabling more reliable multimodal interactions. Additionally, we introduce Rebalanced Modality Gradient Modulation (ReGrad) for adaptive gradient modulation to balance modality convergence. To further enhance generalization, propose Asymmetric Domain Prompts (ADPs) that leverage CLIP’s language priors to learn generalized decision boundaries across modalities. We also develop a novel multimodal FAS benchmark to evaluate generalizability under various deployment conditions. Extensive experiments across this benchmark show our method outperforms state-of-the-art FAS methods, demonstrating superior generalization capability.

源语言英语
页(从-至)7608-7625
页数18
期刊IEEE Transactions on Pattern Analysis and Machine Intelligence
47
9
DOI
出版状态已出版 - 2025

指纹

探究 'Reliable and Balanced Transfer Learning for Generalized Multimodal Face Anti-Spoofing' 的科研主题。它们共同构成独一无二的指纹。

引用此