Prompt and Parameter Co-Optimization for Large Language Models

Xiaohe Bo, Rui Li, Zexu Sun, Quanyu Dai, Zeyu Zhang, Zihang Tian, Xu Chen, Zhenhua Dong

Keywords: prompt-parameter co-optimization, shared-private parameterization, supervised regularization

TL;DR: This paper proposes MetaTuner, a novel framework that jointly optimizes prompts and parameters of LLMs through a knowledge-sharing mechanism and a supervised regularization loss, achieving superior performance across multiple benchmarks.

Abstract:

Prompt optimization and fine-tuning are two major approaches to improve the performance of Large Language Models (LLMs).
They enhance the capabilities of LLMs from complementary perspectives: the former through explicit natural language, and the latter through implicit parameter updates.
However, prior work has typically studied them in isolation, leaving their synergistic potential largely underexplored. To bridge this gap, in this paper, we introduce MetaTuner, a novel framework that jointly integrates prompt optimization and fine-tuning for LLM training.
Specifically, we introduce two neural networks to generate prompts and parameters, respectively, while allowing them to share a common bottom encoding layer to enable knowledge sharing.
By the guidance of the final supervised signals, our framework is optimized to discover the optimal combinations between the prompts and parameters.
Given that prompt learning involves discrete optimization while fine-tuning operates in a continuous parameter space, we design a supervised regularization loss to train our framework effectively.
Extensive experiments across diverse benchmarks show that our method consistently outperforms the baselines. To benefit the research community, we have released our project at https://github.com/BoXiaohe/MetaTuner.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 10243