Textual Entailment in Legal Bar Exam Question Answering Using Deep Siamese Networks

Mi-Young Kim, Yao Lu, Randy Goebel

Abstract: Every day a large volume of legal documents are produced, and lawyers need support for their analysis, especially in corporate litigation. Typically, corporate litigation has the aim of finding evidence for or against the litigation claims. Identifying the critical legal points within large volumes of legal text is time consuming and costly, but recent advances in natural language processing and information extraction have provided new enthusiasm for improved automated management of legal texts and the identification of legal relationships. As a legal information extraction example, we have constructed a question answering system for Yes/No bar exam questions. Here we introduce a Siamese deep Convolutional Neural Network for textual entailment in support of legal question answering. We have evaluated our system using the data from the competition on legal information extraction/entailment (COLIEE). The competition focuses on the legal information processing required to answer yes/no questions from legal bar exams, and it consists of two phases: legal ad-hoc information retrieval (Phase 1), and textual entailment (Phase 2). We focus on Phase 2, which requires “Yes” or “No” answers to previously unseen queries. We do this by comparing the extracted meanings of queries and relevant articles. Our choice of features used for the semantic modeling focuses on word properties and negation. Experimental evaluation demonstrates the effectiveness of the Siamese Convolutional Neural Network, and our results show that our Siamese deep learning-based method outperforms the previous use of a single Convolutional Neural Network.