Enhancement of In-Context Reasoning in LLMs through Inductive Rule Learning
Keywords: In-Context Learning, Inductive Reasoning
Abstract: Currently, Large language models (LLMs) have achieved remarkable performance across various language tasks, largely due to their training on extensive datasets and their considerable model size. These models exhibit in-context learning abilities, which is to learn through few-shot learning. However, the underlying reasoning process remains ambiguous, it is unclear whether the model simply retrieves relevant information and instructions from its training data to generate similar responses, or whether it generalizes examples to form overarching rules, which are then applied to produce accurate answers. Another method for improving few-shot learning is Chain-of-Thought prompting that complement steps by steps instruction for LLMs, so they can follow this instruction to solve many reasoning tasks. Several approaches for evaluating the reasoning abilities of LLMs typically involve task-solving through code generation, which enables models to formalize problems and leverage a code compiler to solve them precisely. However, these methods are constrained to specific task types and are insufficient for a comprehensive assessment of the model's broader reasoning capabilities. Therefore, this paper proposes a method to enhance in-context learning capabilities through two main stages: generating general rules from the provided examples and utilizing LLMs to verify these general rules, thereby aiming to improve reliability and accuracy. At the same time, this approach seeks to investigate the inductive and deductive reasoning abilities, and can improve our understanding of the model’s reasoning by generating and applying general rules to provide transparent, clearly explained responses. The proposed method demonstrates competitive performance on the 1D-ARC benchmark and several traditional language tasks, suggesting its potential for more robust evaluation of LLM reasoning abilities.
Primary Area: generative models
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 9911
Loading