A Two-Stream Word-level Information Fusion BERT Model for Event Element Entity Recognition

Yan Zhang, Kaiwen Li, Jinghui Zhang, Jingya Zhou

Abstract: This paper primarily explores methods for enhancing the performance of event element entity recognition tasks by enriching the positional information of vocabulary and focusing on document context. It proposes and implements an event element entity recognition algorithm based on vocabulary-enhanced dual-stream encoders named Two-Stream Word-level Information Fusion BERT, TSWLIF-BERT. A word-level information fusion layer that fully integrates the vocabulary and its positional information was designed and implemented.: (1) Designed and implemented a word-level information fusion layer that fully integrates both the vocabulary and their positional information, deeply integrating the vocabulary and positional information in the pre-trained model, thereby obtaining an enhanced vocabulary model (Word-level Information Fusion BERT, WLIF-BERT).(2) Based on the model WLIF-BERT, designed a dual-stream encoder that considers cross-sentence information with a global encoder and focuses on sentence-internal information with a local encoder, fully modeling the contextual information.(3) Used masked conditional random fields to solve the problem of illegal sequence label transitions in models based on conditional random fields. Experiments show that this method significantly improves performance in event element entity recognition tasks.

External IDs:dblp:conf/cbd/ZhangLZZ24