Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations

Wei-Lin Chen, Cheng-Kuang Wu, Yun-Nung Chen, Hsin-Hsi Chen

Submission Type: Regular Long Paper

Submission Track: Theme Track: Large Language Models and the Future of NLP

Keywords: in-context learning, zero-shot, bootstrapping

TL;DR: This work presents Self-ICL, a framework bootstrapping LLMs' intrinsic capabilities for zero-shot in-context learning via self-generated demonstrations.

Abstract:

Large language models (LLMs) have exhibited striking in-context learning (ICL) ability to adapt to target tasks with a few input-output demonstrations.
For better ICL, different methods are proposed to select representative demonstrations from existing training corpora.
However, such settings are not aligned with real-world practices, as end-users usually query LMs without access to demonstration pools.
In this work, we introduce Self-ICL---a simple framework which bootstraps LMs' intrinsic capabilities to perform zero-shot ICL.
Given a test input, Self-ICL first prompts the model to generate pseudo-inputs.
Next, the model predicts pseudo-labels for the pseudo-inputs via zero-shot prompting.
Finally, we perform ICL for the test input with the pseudo-input-label pairs as demonstrations.
Evaluation on 23 BIG-Bench Hard tasks shows Self-ICL outperforms zero-shot baselines on both average accuracy and head-to-head comparison.
Moreover, with zero-shot chain-of-thought, Self-ICL achieves results comparable to using real demonstrations.
Additionally, we conduct a range of analyses to validate Self-ICL's effectiveness and provide insights for its behaviors under different settings.

Submission Number: 3894