Hierarchical symmetric cross entropy for distant supervised relation extraction

Yun Liu, Xiaoheng Jiang, Pengshuai Lv, Yang Lu, Shupan Li, Kunli Zhang, Mingliang Xu

Abstract: Distant supervised relation extraction has been increasingly popular in recent years, which generates datasets automatically without human intervention. However, the distant supervised assumption has the limitation that the generated datasets have inevitable labeling errors. This paper proposes the method of Hierarchical Symmetric Cross Entropy for Distant Supervised Relation Extraction (HSCERE) to alleviate the impact of the noisy labels. Specifically, HSCERE simultaneously utilizes two extractors with the same network structure for collaborative learning. This collaborative learning process guides the optimization of the extractor through a joint loss function, namely Hierarchical Symmetric Cross Entropy (HSCE). Within the HSCE loss, the predicted probability distribution of the extractors serves as the supervisory signal, guiding the optimization of the extractors on two levels to reduce the impact of noisy labels. The two levels include the internal optimization within each extractor and the collaborative optimization between extractors. Experiments on generally used datasets show that HSCERE can effectively handle noisy labels and can be incorporated into various methods to enhance their performance.