GENEVA: Pushing the Limit of Generalizability for Event Argument Extraction with 100+ Event Types

Anonymous

Abstract: Event Argument Extraction (EAE) deals with the task of extracting event-specific information from texts. EAE models usually require a large amount of annotated data for training, but procuring annotations is expensive for each new event type. To cater to the emerging event types and new domains in a realistic setting, it is growingly imperative for EAE models to be generalizable. However, most existing EAE benchmark datasets like ACE and ERE have limited diversity and coverage in terms of event types and cannot adequately evaluate the generalizability of EAE models. To alleviate this issue, we introduce GENEVA, a new dataset covering a diverse range of 115 event types and 187 argument role types. We create four benchmarking test suites in GENEVA to assess EAE models' generalizability. Additionally, we propose a new model AutoDEGREE which establishes a strong benchmark on these test suites. Lastly, we evaluate the generalizability of recent EAE systems from different model families and analyze their behaviors on GENEVA.

Paper Type: long

Research Area: Information Extraction