Federated R\'enyi Fair Inference in Federated Heterogeneous System

Zhiyong Ma, Yuanjie Shi, Yan Yan, Jian Chen

Keywords: Fairness, Federated Learning

TL;DR: We propose FedR\'enyi, an algorithm that enhances group fairness in Federated Learning and addresses statistical and system heterogeneity, achieving better accuracy and fairness trade-off than existing methods.

Abstract:

Federated Learning (FL) is a prominent distributed learning approach that addresses two major challenges: statistical heterogeneity (i.e., non-identically distributed data) and system heterogeneity (i.e., variability in the communication and computation on each client).
As FL is commonly applied in sectors such as commercial and financial, group disparities may emerge and cause harm.
However, current fairness algorithms assume homogeneous data, which does not align with the FL context.
The main challenge is estimating global fairness measures (e.g., R\'enyi or Pearson correlation) in an asynchronous, heterogeneous system.
To address this, we propose the FedR\'enyi algorithm, which regularizes fairness by R\'enyi correlation.
For statistical heterogeneity, FedR\'enyi aggregates local fairness statistics to estimate the global R\'enyi correlation with an estimation error bound of $O(1/\sqrt{n})$, where $n$ is the total number of data samples.
This theoretical result improves significantly over the prior result $O(1/\sqrt{K})$ with $K$ clients.
We further prove that FedR\'enyi converges at the same rate as in the homogeneous setting.
For system heterogeneity, FedR\'enyi approximates missing client updates through weighted averaging over a nearest neighbor region, ensuring a non-expansive approximation error under non-convex conditions.
Extensive experiments demonstrate that FedR\'enyi achieves a promising fairness-accuracy trade-off, with at least 2\\% improvement over baselines.

Code Link: https://github.com/AllenMa97/Federated-Renyi

Readers: auai.org/UAI/2025/Conference, auai.org/UAI/2025/Conference/Area_Chairs, auai.org/UAI/2025/Conference/Reviewers, auai.org/UAI/2025/Conference/Submission337/Authors, auai.org/UAI/2025/Conference/Submission337/Reproducibility_Reviewers

Submission Number: 337