Towards Robust Information Extraction via Binomial Distribution Guided Counterpart Sequence

Yinhao Bai, Yuhua Zhao, Zhixin Han, Hang Gao, Chao Xue, Mengting Hu

Abstract: Information extraction (IE) aims to extract meaningful structured tuples from unstructured text. Existing studies usually utilize a pre-trained generative language model that rephrases the original sentence into a target sequence, which can be easily decoded as tuples. However, traditional evaluation metrics treat a slight error within the tuple as an entire prediction failure, which is unable to perceive the correctness extent of a tuple. For this reason, we first propose a novel IE evaluation metric called Matching Score to evaluate the correctness of the predicted tuples in more detail. Moreover, previous works have ignored the effects of semantic uncertainty when focusing on the generation of the target sequence. We argue that leveraging the built-in semantic uncertainty of language models is beneficial for improving its robustness. In this work, we propose <u>B</u>inomial distribution guided <u>c</u>ounterpart <u>s</u>equence (BCS) method, which is a model-agnostic approach. Specifically, we propose to quantify the built-in semantic uncertainty of the language model by bridging all local uncertainties with the whole sequence. Subsequently, with the semantic uncertainty and Matching Score, we formulate a unique binomial distribution for each local decoding step. By sampling from this distribution, a counterpart sequence is obtained, which can be regarded as a semantic complement to the target sequence. Finally, we employ the Kullback-Leibler divergence to align the semantics of the target sequence and its counterpart. Extensive experiments on 14 public datasets over 5 information extraction tasks demonstrate the effectiveness of our approach on various methods. Our code and dataset are available at https://github.com/byinhao/BCS.