ON THE ROBUSTNESS OF CHATGPT UNDER INPUT PERTURBATIONS FOR NAMED ENTITY RECOGNITION TASK

Ishani Mondal, Abhilasha Sancheti

Keywords: Robustness, LLM, Explanations

Abstract:

We present a systematic evaluation of the robustness of ChatGPT (in both zeroand
few-shot settings) under input perturbations for Named Entity Recognition
(NER) task. Our findings suggest: (1) ChatGPT is more brittle on Drug or Disease
entity perturbations (rare entities) as compared to those on widely known Person
or Location entities, and (2) the quality of explanations (localness and globalness)
for the same entity considerably differ under various Entity-specific and Contextspecific
perturbations; the quality significantly improves using in-context learning.

Submission Number: 82