An Empirical Study on Prompt Compression for Large Language Models

Zhang Zheng, Jinyi Li, Yihuai Lan, Xiang Wang, Hao Wang

Track: Long Paper Track (up to 9 pages)

Keywords: prompt compression, explanation faithfulness, feature attribution, robustness

TL;DR: A comprehensive study on different prompt compression methods for LLMs across various tasks and datasets.

Abstract: Prompt engineering enables Large Language Models (LLMs) to perform a variety of tasks. However, lengthy prompts significantly increase computational complexity and economic costs. To address this issue, we study six prompt compression methods for LLMs, aiming to reduce prompt length while maintaining LLM response quality. In this paper, we present a comprehensive analysis covering aspects such as generation performance, model hallucinations, efficacy in multimodal tasks, word omission analysis, and more. We evaluate these methods across 13 datasets, including news, scientific articles, commonsense QA, math QA, long-context QA, and VQA datasets. Our experiments reveal that prompt compression has a greater impact on LLM performance in long contexts compared to short ones. In the Longbench evaluation, moderate compression even enhances LLM performance.

Submission Number: 104