Do Prompt-Based Models Really Understand the Meaning of Their Prompts?

Anonymous

Abstract: Recently, a boom of papers has shown extraordinary progress in zero-shot and few-shot learning with various prompt-based models. Such success can give the impression that prompts help models to learn faster in the same way that humans learn faster when provided with task instructions expressed in natural language. In this study, we experiment with over 30 prompts manually written for natural language inference (NLI). We find that models learn just as fast with many prompts that are intentionally irrelevant or even pathologically misleading as they do with instructively “good” prompts. Further, such patterns hold even for models as large as 175 billion parameters (Brown et al., 2020) as well as the recently proposed instruction-tuned models which are trained on hundreds of prompts (Sanh et al., 2021; Wei et al., 2021). Despite some success, instruction-tuned models are capable of producing good predictions with misleading prompts even at zero shots. In sum, notwithstanding prompt-based models’ impressive improvement, we find evidence of serious limitations that question the degree to which language models really understand the meaning of prompts in the way humans do.

Paper Type: long