UniICL: An Efficient ICL Framework Unifying Compression, Selection, and Generation

ACL ARR 2024 August Submission208 Authors

Abstract:

In-context learning (ICL) enhances the reasoning abilities of Large Language Models (LLMs) by prepending a few demonstrations.
It motivates researchers to introduce more examples to provide additional contextual information for the generation.
However, existing methods show a significant limitation due to the problem of excessive growth in context length which causes a large hardware burden.
Additionally, shallow-relevant examples selected by out-off-shelf tools hinder LLMs from capturing useful contextual information for generation.
In this paper, to approach these limitations, we propose \textbf{UniICL}, a novel \textbf{Uni}fied \textbf{ICL} framework that unifies demonstration compression, demonstration selection, and final response generation.
Furthermore, to avoid repeated compression of the same demonstration and boost inference efficiency, we design a tailored compression strategy that allows UniICL caching compression results into \textbf{Demonstration Bank} (\textbf{DB}).
Extensive out-of-domain evaluations prove the advantages of UniICL in both effectiveness and efficiency.

Paper Type: Long

Research Area: Efficient/Low-Resource Methods for NLP

Research Area Keywords: LLM, ICL,Prompt compression

Contribution Types: Approaches to low-resource settings

Languages Studied: English

Submission Number: 208