The Natural Language Decathlon: Multitask Learning as Question AnsweringDownload PDF

27 Sept 2018 (modified: 22 Oct 2023)ICLR 2019 Conference Blind SubmissionReaders: Everyone
Abstract: Deep learning has improved performance on many natural language processing (NLP) tasks individually. However, general NLP models cannot emerge within a paradigm that focuses on the particularities of a single metric, dataset, and task. We introduce the Natural Language Decathlon (decaNLP), a challenge that spans ten tasks: question answering, machine translation, summarization, natural language inference, sentiment analysis, semantic role labeling, relation extraction, goal-oriented dialogue, semantic parsing, and commonsense pronoun resolution. We cast all tasks as question answering over a context. Furthermore, we present a new multitask question answering network (MQAN) that jointly learns all tasks in decaNLP without any task-specific modules or parameters more effectively than sequence-to-sequence and reading comprehension baselines. MQAN shows improvements in transfer learning for machine translation and named entity recognition, domain adaptation for sentiment analysis and natural language inference, and zero-shot capabilities for text classification. We demonstrate that the MQAN's multi-pointer-generator decoder is key to this success and that performance further improves with an anti-curriculum training strategy. Though designed for decaNLP, MQAN also achieves state of the art results on the WikiSQL semantic parsing task in the single-task setting. We also release code for procuring and processing data, training and evaluating models, and reproducing all experiments for decaNLP.
Keywords: multitask learning, natural language processing, question answering, machine translation, relation extraction, semantic parsing, commensense reasoning, summarization, entailment, sentiment, dialog
TL;DR: We introduce a multitask learning challenge that spans ten natural language processing tasks and propose a new model that jointly learns them.
Code: [![github](/images/github_icon.svg) salesforce/decaNLP](https://github.com/salesforce/decaNLP) + [![Papers with Code](/images/pwc_icon.svg) 4 community implementations](https://paperswithcode.com/paper/?openreview=B1lfHhR9tm)
Data: [decaNLP](https://paperswithcode.com/dataset/decanlp), [CNN/Daily Mail](https://paperswithcode.com/dataset/cnn-daily-mail-1), [MultiNLI](https://paperswithcode.com/dataset/multinli), [QA-SRL](https://paperswithcode.com/dataset/qa-srl), [SNLI](https://paperswithcode.com/dataset/snli), [SST](https://paperswithcode.com/dataset/sst), [WSC](https://paperswithcode.com/dataset/wsc), [WikiSQL](https://paperswithcode.com/dataset/wikisql)
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 6 code implementations](https://www.catalyzex.com/paper/arxiv:1806.08730/code)
24 Replies

Loading