Large Language Models are Demonstration Pre-Selectors for Themselves

Jiarui Jin, Yuwei Wu, Haoxuan Li, Xiaoting He, Weinan Zhang, Yiming Yang, Yong Yu, Jun Wang, Mengyue Yang

TL;DR: Our main contribution is to propose a novel demonstration pre-selector which is different from previous demonstration selectors.

Abstract: In-context learning with large language models (LLMs) delivers strong few-shot performance by choosing few-shot demonstrations from the entire training dataset. However, previous few-shot in-context learning methods, which calculate similarity scores for choosing demonstrations, incur high computational costs by repeatedly retrieving large-scale datasets for each query. This is due to their failure to recognize that not all demonstrations are equally informative, and many less informative demonstrations can be inferred from a core set of highly informative ones. To this end, we propose FEEDER (FEw yet Essential Demonstration prE-selectoR), a novel \emph{pre-selection} framework that identifies a core subset of demonstrations containing the most informative examples. This subset, referred to as the FEEDER set, consists of demonstrations that capture both the ''sufficiency'' and ''necessity'' information to infer the entire dataset. Notice that FEEDER is selected before the few-shot in-context learning, enabling more efficient few-shot demonstrations choosing in a smaller set. To identify FEEDER, we propose a novel effective tree based algorithm. Once selected, it can replace the original dataset, leading to improved efficiency and prediction accuracy in few-shot in-context learning. Additionally, FEEDER also benefit fine-tuning LLMs, we propose a bi-level optimization method enabling more efficient training without sacrificing performance when datasets become smaller. Our experiments are on 6 text classification datasets, 1 reasoning dataset, and 1 semantic-parsing dataset, across 6 LLMs (ranging from 335M to 7B parameters), demonstrate that: (i) In few-shot inference, FEEDER achieves superior (or comparable) performance while utilizing only half the input training data. (ii) In fine-tuning, FEEDER significantly boosts the performance of LLMs.

Lay Summary: Large language models (LLMs), like those behind chatbots and AI assistants, get better at new tasks when shown a few example questions and answers. However, picking the best examples from a huge pile can be slow and computationally expensive. We found that many examples do not add much value — most of the useful information comes from a small, carefully chosen set. To address this, we created FEEDER (FEw yet Essential Demonstration prE-selectoR), a method that selects the smallest set of essential examples needed for the AI to learn as much as possible. FEEDER identifies these core examples before training, using a novel tree-based approach. This makes it much faster and cheaper to pick examples for the model, and actually helps the AI make better predictions. We tested FEEDER across several language tasks and different AIs. We found it can match or beat previous methods while using only half the examples, and it also improves training for smaller datasets. This could help make large language models more efficient and accessible.

Link To Code: https://github.com/GUNDAM-Labet/GUNDAM

Primary Area: Deep Learning->Large Language Models

Keywords: Large Language Model, Demonstration Pre-Selection, In-Context Learning

Submission Number: 12365