Go to ACL ARR 2025 February homepageAutomating Legal Concept Interpretation with LLMs: Retrieval, Generation, and Evaluation
Abstract: Legal articles often include vague concepts for adapting to the ever-changing society.
Providing detailed interpretations of these concepts is a critical and challenging task even for legal practitioners. It requires meticulous and professional annotations and summarizations by legal experts, which are admittedly time-consuming and expensive to collect at scale.
By emulating legal experts' doctrinal method, we introduce a novel framework, **ATRIE**, using large language models (LLMs) to **A**u**T**omatically **R**etrieve concept-related information, **I**nterpret legal concepts, and **E**valuate generated interpretations, eliminating dependence on legal experts.
ATRIE comprises a legal concept interpreter and a legal concept interpretation evaluator. The interpreter uses LLMs to retrieve relevant information from judicial precedents and interpret legal concepts.
The evaluator uses performance changes on legal concept entailment, a downstream task we propose, as a proxy of interpretation quality.
Automatic and multifaceted human evaluations indicate that the quality of our interpretations is comparable to those written by legal experts, with superior comprehensiveness and readability. Although there remains a slight gap in accuracy, it can already assist legal practitioners in improving the efficiency of concept interpretation.
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: Legal NLP
Languages Studied: Chinese
Submission Number: 5959
Loading