SELF: Self-Evolution with Language Feedback
Abstract: Large Language Models (LLMs) have shown impressive adaptability in various fields, yet the optimal pathway of autonomous model evolution remains under-explored.
Drawing inspiration from the self-driven learning process of humans, we introduce \textit{SELF} (Self-Evolution with Language Feedback), a novel learning framework that empowers LLMs to continually self-improve their abilities.
SELF initiates with a meta-skill learning process that equips the LLMs with capabilities for self-feedback and self-refinement.
SELF employs language-based feedback for detailed and nuanced evaluations, pinpointing response flaws and suggesting refinements.
Subsequently, the model engages in an iterative process of self-evolution: they autonomously generate responses to unlabeled instructions, refine these responses interactively, and use the refined and filtered data for iterative self-training, thereby progressively boosting their capabilities.
Moreover, the SELF framework equips the model with the ability to self-refine during inference, leading to further improved response quality.
Our experiments on mathematical and general tasks demonstrate that SELF enables the model to continually self-improve without human intervention. The SELF framework indicates a promising direction for the autonomous evolution of LLMs, transitioning them from passive information receivers to active participants in their development.
Paper Type: Long
Research Area: Generation
Research Area Keywords: Large Language Models, Self-Improvement, Language Feedback
Contribution Types: NLP engineering experiment
Languages Studied: English
Submission Number: 1216
Loading