Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Work
Experimental Settings
Experimental Results
References
All Topics
Computer Science
Machine Learning

Abnormal Client Behavior Detection in Federated Learning

Profile image of Tianjian ChenTianjian Chen

2019, ArXiv

Abstract

In federated learning systems, clients are autonomous in that their behaviors are not fully governed by the server. Consequently, a client may intentionally or unintentionally deviate from the prescribed course of federated model training, resulting in abnormal behaviors, such as turning into a malicious attacker or a malfunctioning client. Timely detecting those anomalous clients is therefore critical to minimize their adverse impacts. In this work, we propose to detect anomalous clients at the server side. In particular, we generate low-dimensional surrogates of model weight vectors and use them to perform anomaly detection. We evaluate our solution through experiments on image classification model training over the FEMNIST dataset. Experimental results show that the proposed detection-based approach significantly outperforms the conventional defense-based methods.

Related papers

Dynamic Federated Learning Model for Identifying Adversarial Clients
M. Victoria Luzón

ArXiv, 2020

Federated learning, as a distributed learning that conducts the training on the local devices without accessing to the training data, is vulnerable to dirty-label data poisoning adversarial attacks. We claim that the federated learning model has to avoid those kind of adversarial attacks through filtering out the clients that manipulate the local data. We propose a dynamic federated learning model that dynamically discards those adversarial clients, which allows to prevent the corruption of the global learning model. We evaluate the dynamic discarding of adversarial clients deploying a deep learning classification model in a federated learning setting, and using the EMNIST Digits and Fashion MNIST image classification datasets. Likewise, we analyse the capacity of detecting clients with poor data distribution and reducing the number of rounds of learning by selecting the clients to aggregate. The results show that the dynamic selection of the clients to aggregate enhances the perfor...

View PDFchevron_right
Towards Mitigating Poisoning Attacks in Federated Learning
Rania Talbi

Le Centre pour la Communication Scientifique Directe - HAL - Inria, 2021

Federated learning is a new machine learning trend that, guided by privacy goals, distributes learning across multiple participants who train the model collaboratively without sharing their data. Nonetheless, it is vulnerable to a variety of attacks such as data and model poisoning. In these attacks, adversaries attempt to inject a backdoor task in the model along with its main task during the training phase. After that, the injected backdoor is exploited at inference-time given a specific trigger. Many state-of-the-art mechanisms that rely on model update auditing have been proposed to mitigate poisoning attacks. We show in this paper that attackers are still capable to evade such detectors by crafting model updates that mimic benign ones. In this paper, we propose ARMOR, a novel mechanism that successfully detects these backdoor attacks in Federated Learning. We describe the design principles of ARMOR based on generative adversarial networks. And we present ARMOR's evaluation results on a real world dataset, which demonstrates that it outperforms its competitors.

View PDFchevron_right
FedDefender: Backdoor Attack Defense in Federated Learning
Waris Gill

arXiv (Cornell University), 2023

Federated Learning (FL) is a privacy-preserving distributed machine learning technique that enables individual clients (e.g., user participants, edge devices, or organizations) to train a model on their local data in a secure environment and then share the trained model with an aggregator to build a global model collaboratively. In this work, we propose FedDefender, a defense mechanism against targeted poisoning attacks in FL by leveraging differential testing. FedDefender first applies differential testing on clients' models using a synthetic input. Instead of comparing the output (predicted label), which is unavailable for synthetic input, FedDefender fingerprints the neuron activations of clients' models to identify a potentially malicious client containing a backdoor. We evaluate FedDefender using MNIST and FashionMNIST datasets with 20 and 30 clients, and our results demonstrate that FedDefender effectively mitigates such attacks, reducing the attack success rate (ASR) to 10% without deteriorating the global model performance. • Computing methodologies → Computer vision tasks.

View PDFchevron_right
Secure Federated Learning against Model Poisoning Attacks via Client Filtering
Duygu Yaldız

arXiv (Cornell University), 2023

Given the distributed nature, detecting and defending against the backdoor attack under federated learning (FL) systems is challenging. In this paper, we observe that the cosine similarity of the last layer's weight between the global model and each local update could be used effectively as an indicator of malicious model updates. Therefore, we propose CosDefense, a cosine-similarity-based attacker detection algorithm. Specifically, under CosDefense, the server calculates the cosine similarity score of the last layer's weight between the global model and each client update, labels malicious clients whose score is much higher than the average, and filters them out of the model aggregation in each round. Compared to existing defense schemes, CosDefense does not require any extra information besides the received model updates to operate and is compatible with client sampling. Experiment results on three real-world datasets demonstrate that CosDefense could provide robust performance under the state-of-the-art FL poisoning attack.

View PDFchevron_right
CONTRA: Defending Against Poisoning Attacks in Federated Learning
Sana Awan

Computer Security – ESORICS 2021, 2021

Federated learning (FL) is an emerging machine learning paradigm. With FL, distributed data owners aggregate their model updates to train a shared deep neural network collaboratively, while keeping the training data locally. However, FL has little control over the local data and the training process. Therefore, it is susceptible to poisoning attacks, in which malicious or compromised clients use malicious training data or local updates as the attack vector to poison the trained global model. Moreover, the performance of existing detection and defense mechanisms drops significantly in a scaled-up FL system with non-iid data distributions. In this paper, we propose a defense scheme named CONTRA to defend against poisoning attacks, e.g., label-flipping and backdoor attacks, in FL systems. CONTRA implements a cosinesimilarity-based measure to determine the credibility of local model parameters in each round and a reputation scheme to dynamically promote or penalize individual clients based on their per-round and historical contributions to the global model. With extensive experiments, we show that CONTRA significantly reduces the attack success rate while achieving high accuracy with the global model. Compared with a state-of-the-art (SOTA) defense, CONTRA reduces the attack success rate by 70% and reduces the global model performance degradation by 50%.

View PDFchevron_right
FedClean: A Defense Mechanism against Parameter Poisoning Attacks in Federated Learning
Vivek Khimani

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

In Federated learning (FL) systems, a centralized entity (server), instead of access to the training data, has access to model parameter updates computed by each participant independently and based solely on their samples. Unfortunately, FL is susceptible to model poisoning attacks, in which malicious or malfunctioning entities share polluted updates that can compromise the model's accuracy. In this study, we propose FedClean, an FL mechanism that is robust to model poisoning attacks. The accuracy of the models trained with the assistance of FedClean is close to the one where malicious entities do not participate.

View PDFchevron_right
A Comprehensive Security Analysis and Comparison of Federated Learning Algorithms: Resilience against Data and Model Poisoning Attacks
Ferdinand KAHENGA

Southern Africa Telecommunication Networks and Applications Conference (SATNAC) 2023, 2023

Deep Learning (DL) has revolutionized the field of artificial intelligence (AI) with its unparalleled ability to analyze images, videos, and unstructured data, surpassing the limitations of traditional Machine Learning (ML) techniques. However, DL's centralized "Gather and Analyze" paradigm introduces communication delays, bandwidth constraints, and privacy compliance issues, particularly in infrastructure-poor settings under emerging privacy regulations. In response to these challenges, Federated Learning (FL) has emerged as a promising solution for training ML models in a distributed and privacy-preserving manner. Nevertheless, the security of FL-aware systems remains a critical concern, particularly in safeguarding against data and model poisoning attacks. This paper aims to shed light on the security of Federated Learning algorithms by performing a comprehensive comparison of three popular FL algorithms, namely FedAVG, FedProx, and FedAdagrad, to evaluate their resilience against data and model poisoning attacks. Our analysis is based on assessing the convergence behavior of these algorithms under attack scenarios. The insights obtained from this paper may pave the way for the development of more robust FL algorithms and promoting the adoption of Federated Learning in real-world applications, where privacy and security are of utmost importance.

View PDFchevron_right
Anomaly Detection from Distributed Data Sources via Federated Learning
Florencia Cavallin

Advanced Information Networking and Applications, 2022

Anomaly detection is an important task to identify rare events such as fraud, intrusions, or medical diseases. However, it often needs to be applied on personal or otherwise sensitive data, e.g. business data. This gives rise to concerns regarding the protection of the sensitive data, especially if it is to be analysed by third parties, e.g. in collaborative settings, where data is collected by different entities, but shall be analysed together to benefit from more effective models. Besides various approaches for e.g. data anonymisation, one approach for privacy-preserving data mining is Federated Learning-especially in settings where data is collected in several distributed locations. A common, global model is obtained by aggregating models trained locally on each data source, while the training data remains at the source. Therefore, data privacy and machine learning can coexist in a decentralised system. While Federated Learning has been studied for several machine learning settings, such as classification, it is still rather unexplored for anomaly detection tasks. As anomalies are rare, they are not picked up easily by a detection method, and the representation in the model dedicated to recognise them might be lost during model aggregation. In this paper, we thus study anomaly detection task on two different benchmark datasets, in supervised, semi-supervised, and unsupervised settings. We federate Multi-Layer Perceptrons, Gaussian Mixture Models, and Isolation Forests, and compare them to a centralised approach.

View PDFchevron_right
Free-riders in Federated Learning: Attacks and Defenses
Jierui Lin

2019

Federated learning is a recently proposed paradigm that enables multiple clients to collaboratively train a joint model. It allows clients to train models locally, and leverages the parameter server to generate a global model by aggregating the locally submitted gradient updates at each round. Although the incentive model for federated learning has not been fully developed, it is supposed that participants are able to get rewards or the privilege to use the final global model, as a compensation for taking efforts to train the model. Therefore, a client who does not have any local data has the incentive to construct local gradient updates in order to deceive for rewards. In this paper, we are the first to propose the notion of free rider attacks, to explore possible ways that an attacker may construct gradient updates, without any local training data. Furthermore, we explore possible defenses that could detect the proposed attacks, and propose a new high dimensional detection method ...

View PDFchevron_right
Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges
M. Victoria Luzón

2022

Federated learning is a machine learning paradigm that emerges as a solution to the privacy-preservation demands in artificial intelligence. As machine learning, federated learning is threatened by adversarial attacks against the integrity of the learning model and the privacy of data via a distributed approach to tackle local and global learning. This weak point is exacerbated by the inaccessibility of data in federated learning, which makes harder the protection against adversarial attacks and evidences the need to furtherance the research on defence methods to make federated learning a real solution for safeguarding data privacy. In this paper, we present an extensive review of the threats of federated learning, as well as as their corresponding countermeasures, attacks versus defences. This survey provides a taxonomy of adversarial attacks and a taxonomy of defence methods that depict a general picture of this vulnerability of federated learning and how to overcome it. Likewise,...

View PDFchevron_right

References (30)

  1. H. Brendan McMahan, Eider Moore, Daniel Ramage, and Blaise Aguera y Arcas. Federated learning of deep networks using model averaging. arXiv preprint uk.arxiv:1602.05629, Feb. 2017.
  2. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Aarti Singh and Jerry Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pages 1273-1282, Fort Lauderdale, FL, USA, 20-22 Apr 2017. PMLR.
  3. Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet Talwalkar. Federated multi-task learning. arXiv preprint arXiv:1705.10467, Feb. 2018.
  4. Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. arXiv preprint arXiv:1902.04885, Feb. 2019.
  5. Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloé Kiddon, Jakub Konecný, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. Towards federated learning at scale: System design. arXiv preprint arXiv:1902.01046, Mar. 2019.
  6. Micah J. Sheller, G. Anthony Reina, Brandon Edwards, Jason Martin, and Spyridon Bakas. Multi-institutional deep learning modeling without sharing patient data: A feasibility study on brain tumor segmentation. CoRR, abs/1810.04304, 2018.
  7. Hao Yu, Sen Yang, and Shenghuo Zhu. Parallel restarted sgd with faster convergence and less communication: Demystifying why model averaging works for deep learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 5693-5700, 2019.
  8. Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization for heterogeneous networks. arXiv preprint arXiv:1812.06127, Jul. 2019.
  9. Xiangyi Chen, Tiancong Chen, Haoran Sun, Zhiwei Steven Wu, and Mingyi Hong. Distributed training with heterogeneous data: Bridging median-and mean-based algorithms. arXiv preprint arXiv:1906.01736, Jun. 2019.
  10. L. T. Phong and T. T. Phuong. Privacy-preserving deep learning via weight transmission. IEEE Transactions on Information Forensics and Security, 14(11):3003-3015, Apr. 2019.
  11. Jiawen Kang, Zehui Xiong, Dusit Niyato, Han Yu, Ying-Chang Liang, and Dong In Kim. Incentive design for efficient federated learning in mobile networks: A contract theory approach. arXiv preprint arXiv:1905.07479, May 2019.
  12. Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for federated learning on user-held data. arXiv preprint arXiv:1611.04482, Nov. 2016.
  13. Robin C. Geyer, Tassilo Klein, and Moin Nabi. Differentially private federated learning: A client level perspective. arXiv preprint arXiv:1712.07557, Mar. 2018.
  14. Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 308-318. ACM, 2016.
  15. Olga Ohrimenko, Felix Schuster, Cédric Fournet, Aastha Mehta, Sebastian Nowozin, Kapil Vaswani, and Manuel Costa. Oblivious multi-party machine learning on trusted processors. In Proceedings of the 25th USENIX Conference on Security Symposium, pages 619-636, 2016.
  16. Yudong Chen, Lili Su, and Jiaming Xu. Distributed statistical machine learning in adversarial settings: Byzantine gradient descent. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 1(2):44, 2017.
  17. Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. In I. Guyon, U. V. Luxburg, S. Ben- gio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 119-129. Curran Associates, Inc., 2017.
  18. Dong Yin, Yudong Chen, Ramchandran Kannan, and Peter Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates. In Jennifer Dy and Andreas Krause, editors, Pro- ceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5650-5659, Stockholmsmässan, Stockholm Sweden, 10-15 Jul 2018. PMLR.
  19. Liping Li, Wei Xu, Tianyi Chen, Georgios B. Giannakis, and Qing Ling. RSA: byzantine-robust stochastic aggregation methods for distributed learning from heterogeneous datasets. arXiv preprint arXiv:1811.03761, Nov. 2018.
  20. Shiqi Shen, Shruti Tople, and Prateek Saxena. Auror: Defending against poisoning attacks in collaborative deep learning systems. In Proceedings of the 32Nd Annual Conference on Computer Security Applications, ACSAC'16, pages 508-519. ACM, Dec. 2016.
  21. Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin B. Calo. Analyzing federated learning through an adversarial lens. arXiv preprint arXiv:1811.12470, Mar. 2019.
  22. Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. arXiv preprint arXiv:1807.00459, Aug. 2019.
  23. Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3):15, 2009.
  24. Mayu Sakurada and Takehisa Yairi. Anomaly detection using autoencoders with nonlinear dimensionality reduction. In Proceedings of the MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis, page 4. ACM, 2014.
  25. John P. Cunningham and Zoubin Ghahramani. Linear dimensionality reduction: Survey, insights, and generalizations. Journal of Machine Learning Research, 16(89):2859-2900, 2015.
  26. Jakub Konecný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492, Oct. 2017.
  27. Yasunori Yamada and Tetsuro Morimura. Weight features for predicting future model perfor- mance of deep neural networks. In IJCAI, pages 2231-2237, 2016.
  28. Sebastian Caldas, Peter Wu, Tian Li, Jakub Konečnỳ, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2018.
  29. Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
  30. Liping Li, Wei Xu, Tianyi Chen, Georgios B Giannakis, and Qing Ling. Rsa: Byzantine- robust stochastic aggregation methods for distributed learning from heterogeneous datasets. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1544-1551, 2019.

Related papers

Using Anomaly Detection to Detect Poisoning Attacks in Federated Learning Applications
kim tran

arXiv (Cornell University), 2022

Adversarial attacks such as poisoning attacks have attracted the attention of many machine learning researchers. Traditionally, poisoning attacks attempt to inject adversarial training data in order to manipulate the trained model. In federated learning (FL), data poisoning attacks can be generalized to model poisoning attacks, which cannot be detected by simpler methods due to the lack of access to local training data by the detector. State-of-the-art poisoning attack detection methods for FL have various weaknesses, e.g., the number of attackers has to be known or not high enough, working with i.i.d. data only, and high computational complexity. To overcome above weaknesses, we propose a novel framework for detecting poisoning attacks in FL, which employs a reference model based on a public dataset and an auditor model to detect malicious updates. We implemented a detector based on the proposed framework and using a one-class support vector machine (OC-SVM), which reaches the lowest possible computational complexity O(K) where K is the number of clients. We evaluated our detector's performance against state-of-the-art (SOTA) poisoning attacks for two typical applications of FL: electrocardiograph (ECG) classification and human activity recognition (HAR). Our experimental results validated the performance of our detector over other SOTA detection methods.

View PDFchevron_right
Chained Anomaly Detection Models for Federated Learning: An Intrusion Detection Case Study
Jan Spooren

Applied Sciences, 2018

The adoption of machine learning and deep learning is on the rise in the cybersecurity domain where these AI methods help strengthen traditional system monitoring and threat detection solutions. However, adversaries too are becoming more effective in concealing malicious behavior amongst large amounts of benign behavior data. To address the increasing time-to-detection of these stealthy attacks, interconnected and federated learning systems can improve the detection of malicious behavior by joining forces and pooling together monitoring data. The major challenge that we address in this work is that in a federated learning setup, an adversary has many more opportunities to poison one of the local machine learning models with malicious training samples, thereby influencing the outcome of the federated learning and evading detection. We present a solution where contributing parties in federated learning can be held accountable and have their model updates audited. We describe a permiss...

View PDFchevron_right
Understanding Federated Learning and Its Application in Video Anomaly Detection
Aniket Singh

International Journal of Engineering Applied Sciences and Technology

Federated Learning is a relatively newer approach, and it tries to solve problems such as reluctance of clients to share data. Instead, a model is sent from server to client(s) where training happens on various edge clients who later update the central model. This is a redesign of the traditional approach of learning from data where all data had to be uploaded to a central server. In contrast, federated learning promotes the idea of bringing code to data which can help overcome latency in timecritical operations. Privacy gives every individual a right to prevent release of data, and several machine learning use-cases deal with such sensitive information for e.g., disease prediction (patient data), anomaly detection (security footage) etc. By introducing the use of federated learning, we can overcome the issue of data sharing and all data will be retained at the edge nodes where the training happens. Through this work we walk through a system design of applying federated learning in a real-time use case of video anomaly detection and present our preliminary results where we achieved an accuracy of 83.34% using the federated paradigm, higher than the centralized accuracy of 81.29%.

View PDFchevron_right
Network-Level Adversaries in Federated Learning
Gökberk Yar

arXiv (Cornell University), 2022

Federated learning is a popular strategy for training models on distributed, sensitive data, while preserving data privacy. Prior work identified a range of security threats on federated learning protocols that poison the data or the model. However, federated learning is a networked system where the communication between clients and server plays a critical role for the learning task performance. We highlight how communication introduces another vulnerability surface in federated learning and study the impact of network-level adversaries on training federated learning models. We show that attackers dropping the network traffic from carefully selected clients can significantly decrease model accuracy on a target population. Moreover, we show that a coordinated poisoning campaign from a few clients can amplify the dropping attacks. Finally, we develop a server-side defense which mitigates the impact of our attacks by identifying and up-sampling clients likely to positively contribute towards target accuracy. We comprehensively evaluate our attacks and defenses on three datasets, assuming encrypted communication channels and attackers with partial visibility of the network.

View PDFchevron_right
Exploration and Exploitation in Federated Learning to Exclude Clients with Poisoned Data
Ala Al-Fuqaha

2022 International Wireless Communications and Mobile Computing (IWCMC), 2022

Federated Learning (FL) is one of the hot research topics, and it utilizes Machine Learning (ML) in a distributed manner without directly accessing private data on clients. However, FL faces many challenges, including the difficulty to obtain high accuracy, high communication cost between clients and the server, and security attacks related to adversarial ML. To tackle these three challenges, we propose an FL algorithm inspired by evolutionary techniques. The proposed algorithm groups clients randomly in many clusters, each with a model selected randomly to explore the performance of different models. The clusters are then trained in a repetitive process where the worst performing cluster is removed in each iteration until one cluster remains. In each iteration, some clients are expelled from clusters either due to using poisoned data or low performance. The surviving clients are exploited in the next iteration. The remaining cluster with surviving clients is then used for training the best FL model (i.e., remaining FL model). Communication cost is reduced since fewer clients are used in the final training of the FL model. To evaluate the performance of the proposed algorithm, we conduct a number of experiments using FEMNIST dataset and compare the result against the random FL algorithm. The experimental results show that the proposed algorithm outperforms the baseline algorithm in terms of accuracy, communication cost, and security.

View PDFchevron_right

Related topics

  • Mathematics
  • Computer Science
  • Anomaly Detection
  • Server Side
  • federated learning
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved