Investigating the Vulnerability of Relation Extraction Models to Semantic Adversarial Attacks

Anonymous

Abstract: In recent years, Large Language Models have set state-of-the-art performance on many NLP tasks.However, these models have been shown to be susceptible to permutations in data, and as such vulnerable to adversarial attacks.In this work, we test the extent of this vulnerability with regards to models fine-tuned for the task of Relation Extraction by generating semantically-close adversarial samples using semantic information on relations, retrieved from an external knowledge base. The results show that fine-tuned models for Relation Extraction are overall affected negatively by adversarials. Our results demonstrate that existing state-of-the-art Relation Extraction models are vulnerable to such adversarial attacks, with performance reductions of up to 33% in F1 score, and with even the most robust model showing a decrease in F1 score of 18%.We also observe that certain patterns arise when the different models face specific permutations, regardless of the architecture implemented.

Paper Type: short

Research Area: Interpretability and Analysis of Models for NLP