PreCoT: Problem Representation Enhances Reasoning in Large Language Models

Seokhyun An, Hyounghun Kim

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Reasoning, Large Language Models, Prompting, Natural Language Processing

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: Chain-of-Thought (CoT) prompting has broken new ground in exploring the reasoning capabilities of large language models (LLMs). Recent studies expand this direction to specific strategies, such as question decomposition and planning, to improve the solution process. On the other hand, within cognitive psychology, problem representation construction is considered a vital aspect of human problem-solving along with the solution process. It involves a solver structurally encoding a problem by defining its initial and goal states, thereby improving the solution process. However, the construction of problem representation has yet to be tapped in further exploring the potential of LLMs' human-like problem-solving ability. In this work, we propose Problem Representation Enhanced CoT (PreCoT), a novel prompting framework that enhances the solution process of LLMs with problem representation. PreCoT is divided into two main stages. First, it extracts the ingredients of the initial and goal state of the problem, which constitute the problem representation together. Next, it initiates an enhanced solution process based on the generated problem representation. In extensive evaluation on benchmarks from a wide range of domains, including arithmetic, commonsense, and symbolic reasoning, PreCoT outperforms CoT on most tasks in both few-shot and zero-shot manners. Additional analyses further demonstrate the effectiveness of problem representation and its contribution to the reasoning in LLMs, as it does in human problem-solving.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5277