DocEE: A Large-Scale and Fine-grained Benchmark for Document-level Event Extraction

Anonymous

Abstract: Event extraction aims to identify an event and then extract the arguments participating in the event. Despite the great success in sentence-level event extraction, events are more naturally presented in the form of documents, with event arguments scattered in multiple sentences. However, a major barrier to promote document-level event extraction has been the lack of large-scale and practical training and evaluation datasets. In this paper, we present DocEE, a new document-level event extraction dataset including 20,000+ events, 100,000+ arguments. We highlight three features: large-scale manual annotations, fine-grained argument types and application-oriented settings. Experiments show that there is still a big gap between state-of-the-art models and human beings (43\% Vs 85\% in F1 score), indicating that DocEE is an open issue. We will publish DocEE upon acceptance.

Paper Type: long