AlignedCoT: Prompting Large Language Models via Native-Speaking Demonstrations

ACL ARR 2024 June Submission4215 Authors

16 Jun 2024 (modified: 02 Aug 2024)ACL ARR 2024 June SubmissionEveryoneRevisionsBibTeXCC BY 4.0
Abstract: Large Language Models prompting, such as using in-context demonstrations, is a mainstream technique for invoking LLMs to perform high-performance and solid complex reasoning (e.g., mathematical reasoning, commonsense reasoning), and has the potential for further human-machine collaborative scientific findings. However, current LLMs are delicate and elusive in prompt words and styles. And there is an unseen gap between LLM understanding and human-written prompts. This paper introduces **AlignedCoT**, an LLM-acquainted prompting technique that includes proficient "native-speaking" in in-context learning for the LLMs. Specifically, it achieves consistent and correct step-wise prompts in zero-shot scenarios by progressively probing, refining, and formatting the LLM chain of thoughts so that free from handcrafted few-shot demonstrations while maintaining the prompt quality. We conduct experiments on mathematical reasoning and commonsense reasoning. We find that LLMs with **AlignedCoT** perform significantly superior to them with human-crafted demonstrations. We further apply **AlignedCoT** for rewriting the GSM8k training set, resulting in a *GSM8k-Align* dataset. We observe its benefits for retrieval augmented generation.
Paper Type: Long
Research Area: Language Modeling
Research Area Keywords: complex reasoning, multi-step reasoning, math reasoning, prompting, in-context learning, chain of thoughts, question answering
Contribution Types: NLP engineering experiment, Data resources
Languages Studied: English
Submission Number: 4215
Loading