GLEN: General-Purpose Event Detection for Thousands of Types

Sha Li, Qiusi Zhan, Kathryn Conger, Martha Palmer, Heng Ji, Jiawei Han

Submission Type: Regular Long Paper

Submission Track: Information Extraction

Keywords: event extraction; event detection; dataset

TL;DR: We introduce a new large-scale event detection dataset GLEN which covers over 3k event types.

Abstract:

The progress of event extraction research has been hindered by the absence of wide-coverage, large-scale datasets.
To make event extraction systems more accessible, we build a general-purpose event detection dataset GLEN, which covers 205K event mentions with 3,465 different types, making it more than 20x larger in ontology than today's largest event dataset. GLEN is created by utilizing the DWD Overlay, which provides a mapping between Wikidata Qnodes and PropBank rolesets. This enables us to use the abundant existing annotation for PropBank as distant supervision.
In addition, we also propose a new multi-stage event detection model specifically designed to handle the large ontology size in GLEN. We show that our model exhibits superior performance compared to a range of baselines including InstructGPT.
Finally, we perform error analysis and show that label noise is still the largest challenge for improving performance for this new dataset.

Submission Number: 2185