FedTOF: Mitigating poisoning attacks in Federated Learning via tail orthogonal forgetting

Federated Learning (FL) is a promising paradigm for distributed model training, enabling clients to collaboratively learn a global model without sharing raw data. However, its decentralized nature exposes FL systems to various attacks, particularly data and model poisoning attacks, which compromise model integrity and performance. Existing defenses typically rely on detecting and removing suspicious clients or updates, which often discard useful information, and malicious updates injected in the early stages cannot be eliminated by client exclusion. To address these challenges, we propose FedTOF, a robust federated defense framework based on tail orthogonal forgetting. Instead of directly excluding suspicious clients, FedTOF identifies and suppresses malicious gradient directions while preserving valuable updates from benign clients. Specifically, we introduce a tail-subspace-based detection mechanism that analyzes the low-energy components of gradient updates to capture abnormal gradient directions indicative of attack behavior. FedTOF then performs orthogonal forgetting, removing the contribution of these attack-related directions from the aggregated gradient while maintaining alignment with the dominant benign update directions. Through extensive experiments, we demonstrate that FedTOF effectively defends against diverse poisoning attacks and mitigates their long-term impact, providing a principled and scalable solution for secure federated learning.