Rule-Guided Extraction: A Hierarchical Rule Optimization Framework for Document-Level Event Argument Extraction

ACL ARR 2025 May Submission6495 Authors

Abstract:

Document-level event argument extraction (EAE) is a critical task in natural language processing.
While most prior approaches rely on supervised training with large labeled datasets or resource-intensive fine-tuning, recent studies explore in-context learning (ICL) with LLMs to reduce data dependence and training costs. However, the performance of ICL-based methods still lags behind fully supervised models.
We highlight a key reason for this shortfall: the lack of sufficient extraction rules.
In this paper,
we conduct a systematic study of using hierarchical rules to enhance LLMs' ICL capabilities.
We first define three types of hierarchical rules and demonstrate their effectiveness in enhancing the performance of LLMs for document-level EAE. Building on this,
we further propose an LLM-driven HiErarchical Rule Optimization (HERO) framework that iteratively generates and selects optimal hierarchical rules. Specifically, in each iteration, high-value instances are selected to produce error feedback, which is used to update and expand hierarchical rule sets. This results in multiple candidate hierarchical rule sets, from which the optimal one is selected using a scoring-based mechanism. During inference, prompts are constructed using the optimal hierarchical rules to enhance ICL performance of LLMs.
Extensive experiments demonstrate the effectiveness of HERO, surpassing few-shot supervised methods and outperforming state-of-the-art prompting baselines by 3.18\% F1 on RAMS, 4.30\% F1 on DocEE-N, and 3.17\% F1 on DocEE-C.

Paper Type: Long

Research Area: Information Extraction

Research Area Keywords: Information Extraction

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 6495