Self-Para-Consistency: Improving Reasoning Tasks at Low Cost for Large Language Models

Anonymous

Abstract: Recently, the self-consistency decoding strategy has shown the ability to improve performance for complex reasoning tasks with large language models (LLMs). However, the costs may be high because the sampling process of the strategy will generate some low-probability text resulting in low-quality reasoning paths. As a consequence, it requires a relatively large sampling number to obtain good aggregation performance. In this paper, we propose an alternative strategy, \emph{self-para-consistency}. It first generates multiple paraphrases for each test question, then generates reasoning paths for the original and all the paraphrased questions based on greedy decoding, and finally selects the most consistent answer. Since all the candidate paths have relatively high probabilities, the sampling number could be much smaller than the self-consistency strategy. Extensive experiments on complex reasoning datasets demonstrate the effectiveness of our method in reducing the sampling number.

Paper Type: short

Research Area: Question Answering

Contribution Types: NLP engineering experiment, Publicly available software and/or pre-trained models

Languages Studied: English