MARS-VFL: A Unified Benchmark for Vertical Federated Learning with Realistic Evaluation

Wei Shen, Weiqi Liu, Mingde Chen, Wenke Huang, Mang Ye

Keywords: Vertical Federated Learning, Distrubuted System, Benchmark

TL;DR: We introduce a unified benchmark for VFL that evaluates efficiency, robustness, and security based on realistic data distributions.

Abstract: Vertical Federated Learning (VFL) has emerged as a critical privacy-preserving learning paradigm, enabling collaborative model training by leveraging distributed features across clients. However, due to privacy concerns, there are few publicly available real-world datasets for evaluating VFL methods, which poses significant challenges to related research. To bridge this gap, we propose MARS-VFL, a unified benchmark for realistic VFL evaluation. It integrates data from practical applications involving collaboration across different features, maintaining compatibility with the VFL setting. Based on this, we standardize the evaluation of VFL methods from the mainstream aspects of efficiency, robustness, and security. We conduct comprehensive experiments to assess different VFL approaches, providing references for unified evaluation. Furthermore, we are the first to unify the evaluation of robustness challenges in VFL and introduce a new method for addressing robustness challenges, establishing standard baselines for future research.

Code URL: https://github.com/shentt67/MARS-VFL

Primary Area: AL/ML data processing and benchmarking Infrastructure (e.g., metrics libraries, visualization libraries, data exploration libraries, distributed data processing solutions, scalable data analysis)

Flagged For Ethics Review: true

Submission Number: 484