Using contradictions to improve QA systems

Anonymous

Keywords: Question Answering, Natural Language Inference, Contradiction, Interpretability

TL;DR: Using contradictions to rank answers for question answering outperforms using entailment and improves the quality of questing answering systems overall.

Abstract: Ensuring the safety of question answering (QA) systems is critical for deploying them in biomedical and scientific domains. One approach to improving these systems uses natural language inference (NLI) to determine whether answers are supported, or entailed, by some background context. However, these systems are vulnerable to supporting an answer with a source that is wrong or misleading. Our work proposes a critical approach by selecting answers based on whether they have been contradicted by some background context. We evaluate this system on multiple choice and extractive QA and find that while the contradiction-based systems are competitive with and often better than entailment-only systems, models that incorporate contradiction, entailment, and QA model confidence scores together are the best. Based on this result, we explore unique opportunities for leveraging contradiction-based approaches such for improving interpretability and selecting better answers.