PerFIT: Personalized Federated Instruction Tuning via Neural Architecture Search

Pengyu Zhang, Yingbo Zhou, Ming Hu, Junxian Feng, Yao Liu, Xian Wei, Mingsong Chen

Keywords: federated instruction tuning, personalized federated learning, neural architecture search

Abstract:

Federated Instruction Tuning (FIT) has shown the ability to enable model instruction tuning among massive data owners without exposing privacy. Yet, it still faces two key challenges, i.e., data and resource heterogeneity. Due to the varying data distribution and preferences among data owners, FIT cannot adapt to the personalized data of individual owners. Moreover, clients with superior computational abilities have to compromise to maintain the same fine-tuning architecture as the weaker clients. Such a constraint prevents the powerful clients from having more trainable parameters for better fine-tuning performances. To address these issues uniformly, we propose a novel Personalized Federated Instruction Tuning (PerFIT) framework based on architecture search. Specifically, PerFIT allows each client to search for a personalized architecture by expanding the trainable parameter space of the global model, pruning them, and obtaining personalized sparse patterns. We further propose personalized parameter-wise aggregation to facilitate flexible aggregation among clients with diverse sparse patterns. This procedure allows personalized instruction fine-tuning within the expanded parameter spaces, concurrently preserving the same number of trainable parameters as the vanilla state, thus introducing no extra resource burden.
The evaluations with multiple LLMs on various instruction-following datasets demonstrate that our approach can achieve up to a 23% decrease in personalized perplexity compared to the state-of-the-art FIT methods.

Primary Area: other topics in machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5616