ChapterCR: A Large-Scale Chapter-level Coreference Resolution Benchmark

Anonymous

Abstract: Coreference Resolution aims to identify mentions that refer to one another in documents. Existing coreference resolution datasets are either small in size or short in coreference chains. To address the issue, we propose ChapterCR, a large-scale chapter-level coreference resolution dataset. In ChapterCR, the coreference chains are longer and there are more distractors between the mention and the right entity, which makes it more challenging. Experiments on ChapterCR show that there is still a large gap between the state-of-art baselines and human beings. Even ChatGPT does not perform very well in ChapterCR, with the F1 score of 74.0\% in ChapterCR-en and 58.8\% in ChapterCR-zh, showing that ChapterCR is still an open problem.

Paper Type: long

Research Area: Resources and Evaluation

Languages Studied: English; Chinese