Are the predictions and explanations provided by ChatGPT on Entity Recognition task robust under Adversarial Attack?

Anonymous

Abstract: In this paper, we assess the robustness (reliability) of ChatGPT under adversarial perturbations for one of the most fundamental tasks of Information Extraction (IE) i.e. Named Entity Recognition (NER). Despite the hype, majority of the researchers have vouched for its language understanding and generation capabilities; a little attention has been paid to understand its robustness: How the input-perturbations affect 1) the predictions, 2) the confidence of predictions and 3) the quality of rationale behind its prediction. We perform a systematic analysis of ChatGPT's robustness (under both zero-shot and few-shot setup) on two NER datasets using both automatic and human evaluation. Based on automatic evaluation metrics, we find that 1) ChatGPT is more brittle on \textbf{Drug} or \textbf{Disease} replacements (rare entities) compared to the perturbations on widely known \textbf{Person} or \textbf{Location} entities, 2) the quality of explanations for the same entity \textit{considerably differ} under different types of \enquote{Entity-Specific} and \enquote{Context-Specific} perturbations and the quality can be significantly improved using in-context learning, and 3) it is overconfident for majority of the incorrect predictions, and hence it could lead to misguidance of the end-users.

Paper Type: long

Research Area: Interpretability and Analysis of Models for NLP

Contribution Types: Model analysis & interpretability

Languages Studied: English

Consent To Share Submission Details: On behalf of all authors, we agree to the terms above to share our submission details.