A Summary of the ALQAC 2024 Competition

Dinh-Truong Do, Son T. Luu, Trang Pham, Trung Vo, Nguyen-Hoang Chu, Quang-Huy Chu, Cuong Nguyen, Minh Nguyen, An Trieu, Dat Nguyen, Thanh Tran, Cong Nguyen, Hiep Nguyen, Chau Nguyen, Nguyen-Khang Le, Dieu-Hien Nguyen, Binh Dang, Phuong Nguyen, Ha-Thanh Nguyen, Vu Tran et al. (1 additional authors not shown)

Abstract: This paper presents an overview of the fourth edition of the Automated Legal Question Answering Competition (ALQAC 2024), held in association with the KSE 2024 conference. The primary objective of ALQAC is to advance the state-of-the-art in processing legal texts for low-resource languages, with a focus on Vietnamese statute law documents. This year, the competition introduces new challenges to foster innovation and collaboration, including scenario-based queries that simulate real-world legal problem-solving and the provision of both expertverified and non-expert-verified training data. ALQAC 2024 features two tasks: Legal Document Retrieval (Task 1) and Legal Question Answering (Task 2). Task 1 requires participants to develop systems capable of retrieving relevant legal articles based on given queries, evaluated using the Macro-F2 score. Task 2 involves creating systems that can accurately answer various types of legal questions, including the newly introduced scenario-based queries, with expert-judged accuracy as the primary evaluation metric. The competition saw increased participation, with 8 teams submitting a total of 36 runs across both tasks. This paper provides a detailed description of the tasks, datasets, and evaluation metrics, followed by an analysis of the approaches adopted by the participating teams. The results and insights gained from this year's competition are discussed, highlighting the significant advancements in legal question answering and document retrieval for low-resource languages.