Reducing Short Circuits in Multiple-Choice Natural Language Reasoning Models with Data Augmentation

Anonymous

Keywords: nature language processing, commonsense reasoning, short circuit, data augmentation, interpretability and analysis of models for NLP

Abstract: Statistical biases in the training data may lead to fragility in neural models that makes choices in multiple-choice natural language reasoning problems without referring to the context or premises. To encourage the models to pay more attention to the relations between the premise and the choices, we propose two biologically inspired operations that can generate new training data that ``forces'' the model to look at the premises and reducing short circuits. They can augment any type of multiple choice reasoning dataset, and can be applied to any supervised learning models. Results show that models trained with the augmented data become more robust against both stress test and original test.

Paper Type: long