ICLR: Iterative Optimization for Information Extraction on In-Context Learning via Rule Filtering

Haoliang Liu, Chengkun Cai, Xu Zhao, Han Zhu, Shizhou Huang, Xinglin Zhang, Tao Chen, Jenq-Neng Hwang, Lei Li

Keywords: In-Context Learning, Named Entity Recognition, Rule Optimization, Iterative Filtering, Large Language Models

TL;DR: We propose ICLR, a control-theoretic framework that optimizes in-context learning rules for information extraction, achieving up to 10% gains without training.

Abstract: Existing information extraction (IE) tasks, such as named entity recognition (NER) and relation extraction (RE), typically rely on fine-tuning or few-shot learning methods. In few-shot learning, large language models (LLMs) demonstrate excellent performance through in-context learning (ICL), which involves guiding the model by providing a few examples or rules in the prompt. However, a major challenge with this approach is the selection and optimization of contextual information for diverse IE tasks. In this work, we introduce ICLR (Iterative Context Learning Rule), a control-theoretic framework that models rule optimization as an adaptive filtering problem for comprehensive information extraction. We treat rules as controllable state variables and design an observer system to monitor and control LLM behavior indirectly, without modifying model parameters. Our method iteratively estimates and updates the optimal rule combinations using performance feedback, thereby reformulating the traditionally complex problem of LLM control into a well-defined state-space optimization that generalizes across multiple IE tasks. We evaluate ICLR on both NER datasets (CoNLL03, ACE05, GENIA) and RE datasets (NYT, CoNLL04), demonstrating rapid convergence and superior performance with minimal training data requirements. Our approach achieves up to 10\% performance improvement over state-of-the-art ICL methods while requiring no additional model training and ICLR provides the first control-theoretic foundation for understanding and optimizing in-context learning behavior in information extraction.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 10231