A Script Event Prediction Method Based on Multi-level Joint Pretraining and Prompt Fine-Tuning

Zhen Chen, Wenzhong Yang, Fuyuan Wei, Zhonghua Wu

Abstract: The script event prediction task aims to predict subsequent events based on contextual events by modeling inter-event relationships. Recent studies have improved inference through event-centric pretraining and external knowledge integration. However, the diversity of script scenarios makes it challenging to obtain suitable knowledge resources, and existing pretrained language models often overlook argument-level correlations, limiting their ability to capture finer-grained relationships within events. To address these issues, we propose a script event prediction method combining multi-level joint pretraining with prompt-based fine-tuning. Our approach embeds event and argument knowledge into the model using a multi-level blank-filling strategy during pretraining. In the fine-tuning stage, we leverage prompt learning within a likelihood-based contrastive loss framework, enabling the model to autonomously mine script-related knowledge without relying on external resources or additional scoring layers. Experimental results on the Multiple Choice Narrative Cloze (MCNC) task demonstrate that our method surpasses state-of-the-art baselines, validating its effectiveness and adaptability.