FedSecurity: A Benchmark for Attacks and Defenses in Federated Learning and Federated LLMs

Shanshan Han, Baturalp Buyukates, Zijian Hu, Han Jin, Weizhao Jin, Lichao Sun, Xiaoyang Wang, Wenxuan Wu, Chulin Xie, Yuhang Yao, Kai Zhang, Qifan Zhang, Yuhui Zhang, Salman Avestimehr, Chaoyang He

Primary Area: datasets and benchmarks

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Security, attack, defense, byzantine, backdoor, federated learning, LLM

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: a benchmark on attacks and defenses in federated learning

Abstract:

This paper introduces FedSecurity, an end-to-end benchmark designed to simulate adversarial attacks and corresponding defense mechanisms in Federated Learning (FL). FedSecurity comprises two major components: FedAttacker, which simulates attacks injected during FL training, and FedDefender, which simulates defensive mechanisms to mitigate the impacts of the attacks. FedSecurity is open-
source and can be customized to cover a wide range of machine learning models (e.g., Logistic Regression, ResNet, and GAN) and federated optimizers (e.g., FedAVG, FedOPT, and FedNOVA). We also demonstrate the use of FedSecurity during federated training of Large Language Models (LLMs), showcasing its adaptability and applicability in more complex scenarios.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1595