Prompt Engineering 101 Prompt Engineering Guidelines from a Linguistic Perspective

Wenjuan Han, Xiang Wei, Xingyu Cui, Ning Cheng, Guangyuan Jiang, Weinan Qian, Chi Zhang

Abstract: Deploying tuning-free prompting is challenging in engineering practice: it not only requires users to engage in cumbersome trials and errors but is also extremely time-consuming, as even a slight change in wording and phrasing could have a huge impact on the final performance. To further investigate the impact of different prompts, in this work, we perform a systematic inspection of four factors in linguistics involved in prompt engineering: syntax, semantics, lexicon, and pragmatics. The empirical results quantify the sensitivity of the output to small textual perturbations in four linguistic factors of prompts. Based on the analysis of these four factors, we present a series of design guidelines to help human users write effective prompts. Human evaluation on amateurs shows that using the proposed guidelines helps humans produce prompts with significant gains in zero-shot performance in Pre-trained Language Models (PLMs) and hence validates the utility of the guidelines.