DynFed: Dynamic Test-Time Adaptation for Federated Learning with Adaptive Rate Networks
Abstract: Test-Time Personalized Federated Learning (TTPFL) has emerged as a promising approach for adapting models to distribution shifts in federated learning (FL) environments without relying on labeled data during testing. However, existing methods often struggle with heterogeneous shifts across clients and lack the flexibility to handle diverse distribution changes effectively. In this paper, we introduce \our, a novel algorithm that dynamically optimizes test-time adaptation (TTA) in FL scenarios with heterogeneous distribution shifts.
Our method leverages Adaptive Rate Networks (ARNs) {\color{blue}(or more generally, adaptive rate functions)} to generate client-specific adaptation rates, enabling more effective handling of diverse shift types, including label skew and feature shifts. \our employs an innovative iterative adaptation process, where {\color{blue}adaptation rates are continuously refined based on the current adaptation state using the ARN function}, without direct access to raw client data. Crucially, we uncover a fundamental dichotomy: optimal adaptation strategies for one-type and multi-type distribution shifts can be diametrically opposed. \our navigates this challenge by automatically adjusting its approach based on the nature of the encountered shifts.
{\color{blue}Extensive experiments demonstrate that \our significantly outperforms existing TTPFL and TTA methods across various shift scenarios. Our theoretical analysis provides convergence and generalization guarantees for our approach and justifies the need for adaptive mechanisms.} Our method shows particularly robust performance in complex multi-type shift environments, where previous approaches often struggle. This work opens new avenues for adaptive and resilient FL in real-world applications where distribution shifts are diverse and unpredictable.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: Dear Editors and Reviewers,
Thank you for your time and insightful feedback on our manuscript, "DynFed: Dynamic Test-Time Adaptation for Federated Learning with Adaptive Rate Networks" . We appreciate the constructive comments, which have helped us significantly improve the clarity and rigor of our paper.
We have carefully considered all points raised by the reviewers and have revised the manuscript accordingly. Below, we provide a point-by-point response to the comments. Revisions in the manuscript are substantial and integrated throughout, but key changes related to specific comments are highlighted below.
Assigned Action Editor: ~Ilan_Shomorony1
Submission Number: 4028
Loading