DynFed: Dynamic Test-Time Adaptation for Federated Learning with Adaptive Rate Networks

TMLR Paper5462 Authors

Abstract:

Test-Time Personalized Federated Learning (TTPFL) has emerged as a promising approach for adapting models to distribution shifts in federated learning (FL) environments without relying on labeled data during testing. However, existing methods often struggle with heterogeneous shifts across clients and lack the flexibility to handle diverse distribution changes effectively. In this paper, we introduce DynFed, a novel algorithm that dynamically optimizes test-time adaptation (TTA) in FL scenarios with heterogeneous distribution shifts. Our method leverages Adaptive Rate Networks (ARNs) to generate client-specific adaptation rates, enabling more effective handling of diverse shift types, including label skew and feature shifts. DynFed employs an innovative iterative adaptation process, where adaptation rates are continuously refined based on the current adaptation state using the ARN function, without direct access to raw client data. Crucially, we uncover a fundamental dichotomy: optimal adaptation strategies for one-type and multi-type distribution shifts can be diametrically opposed. DynFed navigates this challenge by automatically adjusting its approach based on the nature of the encountered shifts. Extensive experiments demonstrate that DynFed significantly outperforms existing TTPFL and TTA methods across various shift scenarios. Our theoretical analysis provides convergence and generalization guarantees for our approach and justifies the need
for adaptive mechanisms. Our method shows particularly robust performance in complex multi-type shift environments, where previous approaches often struggle. This work opens new avenues for adaptive and resilient FL in real-world applications where distribution shifts are diverse and unpredictable.

Submission Length: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=6Av7iP4AXc&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DTMLR%2FAuthors%23your-submissions)

Changes Since Last Submission: The added theoretical proofs can demonstrate the validity of our approach and the implications for federated learning on test-time adaption scenarios.

Assigned Action Editor: ~Zachary_B._Charles1

Submission Number: 5462