COLIEE 2022 Summary: Methods for Legal Document Retrieval and Entailment

Mi-Young Kim, Juliano Rabelo, Randy Goebel, Masaharu Yoshioka, Yoshinobu Kano, Ken Satoh

Abstract: We present a summary of the 9th Competition on Legal Information Extraction and Entailment (COLIEE 2022). The competition consists of four tasks on case law and statute law. The case law component includes an information retrieval task (Task 1), and the confirmation of an entailment relation between an existing case and an unseen case (Task 2). The statute law component includes an information retrieval task (Task 3) and an entailment/question answering task (Task 4). Participation was open to any group, using any approach. Ten different teams participated in the case law competition tasks, most of them in more than one task. We received competition submissions from 9 teams for Task 1 (26 runs) and 5 teams for Task 2 (14 runs). On the statute law task, there were 11 different teams participating, most in more than one task. Five teams submitted a total of 15 runs for Task 3, and 6 teams submitted a total of 18 runs for Task 4. We summarize the technical details of all approaches, describe our official evaluation, and provide an overall analysis on our data and submission results.