Empowering Legal Citation Recommendation via Efficient Instruction-Tuning of Pre-trained Language Models

Jie Wang, Kanha Bansal, Ioannis Arapakis, Xuri Ge, Joemon M. Jose

Abstract: The escalating volume of cases in legal adjudication has amplified the complexity of citing relevant regulations and authoritative cases, posing an increasing challenge for legal professionals. Current legal citation prediction methods, which are predominantly reliant on keyword or interest-based retrieval, are proving insufficient. In particular, Collaborative Filtering (CF) based legal recommendation methods exhibited low accuracy. In response to these challenges, we propose the Instruction GPT with Low-Rank Adaptation architecture (IGPT-LoRA), aiming to enhance the performance of legal citation recommendations and reduce computational demands by tuning Pre-trained Language Models (PLMs). IGPT-LoRA leverages prompting and efficient tuning strategies, thus offering a significant improvement over previous context-aware legal citation prediction methods. We design effective domain-specific instruction templates to guide the adaptation of PLMs for recommendation purposes, shedding light on the potential of prompt-based learning in the legal domain. Furthermore, we optimize the learning process with an efficient tuning layer - the Low-Rank Adaptation (LoRA) architecture - to bolster applicability. Experimental results on a real-world legal data set (BVA) demonstrate that IGPT-LoRA outperforms state-of-the-art methods, delivering substantial improvements in accuracy and also in training time and computational efficiency.