A Survey of Attack and Defense Techniques for Federated Learning Systems

As an emerging technology of building machine learning (ML) model using distributed training data sets, federated learning (FL) can effectively solve the problem of local data privacy disclosure caused by joint modeling between different data owners. Therefore, it is widely used in many fields and has developed rapidly. FL keeps the data of participants local and only uploads model parameters to the server, which effectively protects the privacy of local data. However, the existing FL systems have been proved to have potential threats in the data collection stage, training stage and inference stage, which endanger the privacy of data and the robustness of the system. In the data collection stage and training stage, attackers may poison the training data or the model, thereby endangering the security of the system. In the inference stage, attackers may input samples to add minor malicious perturbations, causing the classifier to incorrectly classify the sample process with a very high probability, which will lead to privacy disclosure. Most of the existing research work describes attack and defense methods in ML, which are not necessarily applicable to FL models, and only focusses on a few attack threats and traditional defenses, lacking a detailed and comprehensive overview of the cutting-edge defenses. Starting with two kinds of potential threats: security threat and privacy threat, we give a detailed definition of security attributes in FL scenarios around confidentiality, integrity and availability (CIA triplet), and summarize various attack methods and defense means in FL systematically and comprehensively. Firstly, we summarize the horizontal and vertical federated learning (VFL) process and potential threats respectively, and analyze the basic concepts, implementation stages and existing schemes of common attacks such as poisoning attack, sample attack and inference attack from the perspectives of antagonistic attack and non-antagonistic attack. Adversarial attacks include poisoning attacks, adversarial sample attacks, free-riding attacks, Sybil attacks, and attacks against communication bottlenecks. Non-adversarial attacks include model extraction attacks, inference attacks, and GAN-based attacks. Further, according to different attack methods, defense means are divided into two categories: robustness enhancement methods and privacy enhancing technologies. The robustness enhancement methods mainly defend against antagonistic attacks, including data sanitization, robustness aggregation, anomaly detection, countermeasure training, knowledge distillation, pruning and other methods. The privacy enhancing technology mainly defends the system against non-antagonistic attacks, including homomorphic encryption, secure multi-party computing, differential privacy and blockchain. And the schemes related to robustness enhancement methods and privacy enhancement techniques in FL are sorted out and summarized. Finally, the paper gives future research direction of robustness and privacy in FL: (1) Establish a secure and stable attack detection and evaluation model, endow FL system with self inspection and evaluation capabilities, and provide real-time protection for internal and external environments; (2) Analyze and infer all possible potential attacks and privacy issues, and build a perfect security attack and defense system based on security encryption technology; (3) Study the unique attack and defense in VFL to solve the bottleneck problem of VFL in practical application; (4) Explore the conflict between robustness and privacy in FL to promote large-scale applications.