Iterative LLM Augmentation through Criteria-Based Feedback

ACL ARR 2025 February Submission2768 Authors

Abstract: Large Language Model (LLM) self-reflection involves an LLM reviewing its past outputs to enhance future responses without relying on external data. This concept has been explored in frameworks like those by Shinn and Madaan. However, challenges remain, as Huang.point out the risk of performance degradation due to overly generic reflective prompts. To address these issues, we introduce a vector-based retrieval framework. Our approach demonstrates significant improvements in decision-making, reasoning, and mathematics tasks, surpassing baseline models like Llama3.2-3b and the SELF-REFINE framework. These results emphasize the potential of targeted self-reflection to improve LLM performance while mitigating common drawbacks. Meanwhile,beyond this method, we also explored the possibility of using a multi-agent approach with auxiliary models to assist in reflection. We trained a model to replace the base model in generating criteria and systematically evaluated the impact of the auxiliary model on the output capability of the self-reflection framework.

Paper Type: Long

Research Area: Language Modeling

Research Area Keywords: large language models, self-reflection, self-correction

Languages Studied: English

Submission Number: 2768