Classification or Generation? Understanding Paradigm Shift for Knowledge-Intensive Tasks

Anonymous

Keywords: Natural Language Processing, Language Modeling, Entity Retrieval, Autoregressive Generation

Abstract: Knowledge-intensive tasks such as entity retrieval are challenging for even cutting edge NLP models since they require models to apply knowledge about the world. Previous studies typically treat this task as classification. Recently, a new paradigm has emerged, which reformats knowledge-intensive tasks as natural language generation. This post summarizes the paradigm shift and reviews the new generative methodology for the ICLR community, providing philosophical questions and new directions.

ICLR Paper: https://openreview.net/forum?id=5k8F6UU39V