Prune to Fit: Enabling Federated Fine-Tuning within Edge Memory Budgets

Yebo Wu, Jingguang Li, Chunlin Tian, Zhijiang Guo, Li Li

Keywords: Federated Fine-Tuning, Memory Efficient, Layer Pruning

TL;DR: We propose FedPruner, an innovative federated fine-tuning paradigm that tackles memory constraints of participating devices via intelligent layer pruning.

Abstract: Federated fine-tuning enables privacy-preserving Large Language Model (LLM) adaptation, but its high memory cost limits participation from resource-constrained devices. We propose FedPruner, an innovative federated fine-tuning paradigm that tackles this via intelligent layer pruning. FedPruner flexibly prunes the global model, creating personalized submodels based on device memory constraints. It employs a macro-micro synergistic pruning framework: a macro-level functionality-driven layer orchestration mechanism groups layers, while a micro-level importance-aware layer selection strategy prunes within groups to build device-specific submodels. We further introduce a fine-grained variant that independently prunes Multi-Head Attention and Feed-Forward Network components to precisely preserve critical architectural elements. Extensive experiments demonstrate that FedPruner significantly outperforms state-of-the-art methods with average accuracy gains of up to 11.11\%. Moreover, it maintains strong robustness under varying memory constraints, yielding a 1.98\% average performance improvement while reducing peak memory usage by 75\%.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 1848