Towards Multi-Domain Chinese Document VQA: a New Dataset and Baseline Method

Liang Qiao, Can Li, Huiyan Wu, Xinglong Lu, Yunlu Xu, Yi Niu, Jiang Zhu, Xi Li

Keywords: Benchmark, Dataset, Chinese DocVQA, In-Context Learning

Abstract: Document Visual Question Answering (DocVQA) remains a significant challenge in the field of document understanding and is a critical evaluation metric for current general-purpose large model techniques. However, prevailing public datasets are predominantly designed for single scenarios or specific sources. Furthermore, most available datasets are in English, limiting the verification of model performance in other languages. This paper presents a novel multi-domain Chinese document VQA dataset, which includes 39 document types from 7 different domains. The designed question set encompasses both common extractive questions and complex abstractive questions. Based on this dataset, we conducted a comprehensive review and analysis of various technical paradigms, including both traditional and large model-based approaches. Using the popular in-context learning framework, we propose a robust baseline that achieves commendable few-shot adaptation. Comparative evaluations demonstrate the superior performance of the proposed method across different solution paradigms. The dataset and code will be published.

Primary Area: datasets and benchmarks

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6725