Leveraging Large Language Models and Cross-Attention Mechanism for Zero-Shot Relation Extraction with Contrastive Learning

ACL ARR 2025 May Submission2007 Authors

Abstract: In the zero-shot relation extraction (ZSRE) task, large language models (LLMs) have shown remarkable capabilities in predicting unknown relations, offering significant improvements in efficiency and flexibility over traditional methods. However, the probabilistic nature of the generation process in LLMs may lead to the occurrence of hallucinations, causing inaccurate relation triples be generated. To relieve this problem, this paper proposes a novel model, Cross-Attention Contrastive Relation Extraction (CACRE), which aims at detecting erroneous relation triples generated by LLMs and then effectively distinguishing valid ones. The CACRE model leverages contrastive learning and cross-attention mechanisms. Specifically, contrastive learning is applied to distinguish between positive and negative relation triples, enhancing the model’s feature extraction capability by learning discriminative features. Subsequently, a cross-attention mechanism is employed to capture the semantic associations between texts and triples, thereby improving the model’s ability to understand and extract information from the input content. Experimental results on the DuIE2.0 Chinese dataset demonstrate that CACRE significantly outperforms baseline models in zero-shot scenario with an average 12\% improvement in precision.

Paper Type: Long

Research Area: Information Extraction

Research Area Keywords: relation extraction, LLMs, zero-shot, contrastive learning

Contribution Types: NLP engineering experiment, Data analysis

Languages Studied: Chinese

Submission Number: 2007