Robust Text Classification: Analyzing Prototype-Based Networks

Anonymous

TL;DR: We analyze the robustness properties of prototype-based networks in text classification tasks while considering different design choices, backbones, and dataset complexities.

Abstract: Downstream applications often require text classification models to be accurate, robust, and interpretable. While the accuracy of the state-of-the-art (large) language models approximates human performance, they are not designed to be interpretable and often exhibit a drop in performance on noisy data in the real world. This lack of robustness is particularly concerning in critical domains; e.g., toxicity detection in social media, where it may harm readers by, for example, misidentifying harmful content. A potential solution can be the family of Prototype-Based Networks (PBNs) that classifies examples based on their similarity to prototypical examples of a class (prototypes) and is natively interpretable and shown to be robust to noise, which enabled its wide usage for computer vision tasks. In this paper, we study whether the robustness properties of PBNs transfer to text classification tasks. We design a modular and comprehensive framework for studying the robustness of PBNs, which includes different backbone architectures, distance functions, and objective functions. The proposed evaluation protocol assesses the robustness of models against character-, word-, and sentence-level perturbations. Our experiments show that PBNs consistently enhance the robustness of vanilla language models, supported by the objective function that keeps prototypes interpretable.

Paper Type: long

Research Area: Interpretability and Analysis of Models for NLP

Contribution Types: Model analysis & interpretability

Languages Studied: English