Evaluating the impact of incorporating ’legalese’ definitions and abstractive summarization on the categorization of legal cases by their holdings

Shiu Tin Ivan Ko, Daniela Virginia Cortes Bermudez, Henry Han, Huiyun Zhang

Keywords: NLP, AI, Deep learning, GPT, BERT

TL;DR: Evaluating the impact of incorporating ’legalese’ definitions and abstractive summarization on the categorization of legal cases

Abstract: Legal text is difficult to understand and requires domain-specific knowledge to read. This work aims to investigate the effect that model stacking and input processing have on information fidelity with the motivation to explore possibilities of expanding the accessibility of legal texts. We developed a legal dictionary through the United States Courts’ Glossary of Legal Terms to map complex terms into simple English and used FLAN-T5 to summarize observations. To evaluate performance, we used binary text classification to predict case holdings using LLMs (Large Language Models) and evaluated the results with and without model pretraining. To assess information fidelity, we ask: "Does model stacking affect classification performance?" and "Does performance change with pretraining?"

Submission Type: Archival (to be published in the Journal of LatinX in AI (LXAI) Research)

Submission Number: 12