FedOne: Query-Efficient Federated Learning for Black-box Discrete Prompt Learning

Ganyu Wang, Jinjie Fang, Maxwell Juncheng Yin, Bin Gu, Xi Chen, Boyu Wang, Yi Chang, Charles Ling

Abstract:

Black-Box Discrete Prompt Learning (BDPL) is a prompt-tuning method that optimizes discrete prompts without accessing model parameters or gradients, making the prompt tuning on a cloud-based Large Language Model (LLM) feasible.
Adapting Federated Learning (FL) to BDPL could further enhance prompt tuning performance by leveraging data from diverse sources.
However, all previous research on federated black-box prompt tuning had neglected the substantial query cost associated with the cloud-based LLM service.
To address this gap, we conducted a theoretical analysis of query efficiency within the context of federated black-box prompt tuning. Our findings revealed that degrading FedAvg to activate only one client per round, a strategy we called \textit{FedOne}, enabled optimal query efficiency in federated black-box prompt learning.
Building on this insight, we proposed the FedOne framework, a federated black-box discrete prompt learning method designed to maximize query efficiency when interacting with cloud-based LLMs.
We conducted numerical experiments on various aspects of our framework, demonstrating a significant improvement in query efficiency, which aligns with our theoretical results.

Lay Summary:

Large language models like ChatGPT are often accessed through paid services that don’t let users see or change the model's internal components. To customize these models for specific tasks, users must repeatedly "query" them, which is both costly and slow.

This paper explores how many users can work together to fine-tune these models without sharing their data, using a method called federated learning. But in this setup, the cost multiplies: each participating user has to make many queries to the LLMs. Making it impractical.

We introduce FedOne, a new approach that trains the model by activating only one user at a time. Our analysis shows that this setup is not only far more efficient in reducing expensive queries but also retains strong performance. We tested this idea on real-world tasks using models like GPT-3.5 and showed that FedOne is both effective and cost-efficient.

FedOne makes it easier for people and organizations to adapt powerful AI tools to their needs at a lower cost.

Link To Code: https://github.com/GanyuWang/FedOne-BDPL

Primary Area: Optimization->Large Scale, Parallel and Distributed

Keywords: Federated Learning, Discrete Prompt Learning, Optimization, Query Efficiency

Submission Number: 2315