Set-Aligning Fine-tuning Framework for Auto-Regressive Event Temporal Graph Generation

Anonymous

Abstract: Event temporal graphs have been shown as convenient and effective representations of complex temporal relations between events in text. While traditional methods are based on a pipeline approach, i.e., event extraction and relation classification, the recently proposed contextualised graph generation methods have shown promising results by employing pre-trained language models to generate linearised graphs as autoregressively. However, this inevitably led to sub-optimal graph generation as the linearised graphs exhibit set characteristics which are instead treated sequentially by language models. This is due to their conventional text generation objectives which end up mistakenly penalizing correct predictions only because of the misalignment between elements in text. In this work, we reformulate the task as a conditional set generation problem, proposing a Set-aligning Fine-tuning Framework allowing smooth employment of large language models. A comprehensive experimental assessment has shown that our proposed framework significantly benefits the event temporal graph generation, and outperforms existing baselines. We further demonstrate that under the zero-shot settings, the structural knowledge introduced through the proposed framework has a significant beneficial impact on model generalisation when the examples available are limited.

Paper Type: long

Research Area: Information Extraction

Contribution Types: NLP engineering experiment

Languages Studied: English