Automating Legal Concept Interpretation with LLMs: Retrieval, Generation, and Evaluation

ACL ARR 2024 December Submission2211 Authors

Abstract:

Legal articles often include vague concepts to adapt to the ever-changing society.
Providing detailed interpretations of these concepts is a critical task for legal practitioners, which requires meticulous and professional annotations by legal experts, admittedly time-consuming and expensive to collect at scale.
In this paper, we introduce a novel retrieval-augmented generation framework, \textbf{ATRI}, for \underline{\textbf{A}}u\underline{\textbf{T}}omatically \underline{\textbf{R}}etrieving relevant information from past judicial precendents and \underline{\textbf{I}}nterpreting vague legal concepts.
We further propose a new benchmark, Legal Concept Entailment, to automate the evaluation of generated concept interpretations without expert involvement.
Automatic evaluations indicate that our generated interpretations can effectively assist large language models (LLMs) in understanding vague legal concepts. Multi-faceted evaluations by legal experts indicate that the quality of our concept interpretations is comparable to those written by human experts.
Our work has strong implications for leveraging LLMs to support legal practitioners in interpreting vague legal concepts and beyond.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: Legal NLP

Contribution Types: NLP engineering experiment, Data resources

Languages Studied: Chinese

Submission Number: 2211