[AML]Self-reflection like Humans, Editable-LLM (E-LLM) is All You Need
Keywords: Editable-LLM, Reinforcement learning, Supervisory signal design
Abstract: We have innovatively designed an Editable-LLM that can constantly reflect and modify the generated content in real time, just like the human reflective process. To be more precise, we add a check mechanism based on the traditional generative large model, which implements the operation of adding, deleting, correcting and checking the generated text. The supervisory signal is provided by the text quality score after the simulation modification is completed just like Reinforcement Learning from Human Feedback(RLHF). However, different from traditional RLHF research, our focus is not to select the best from multiple outputs, but to guide LLM to improve a rough draft step by step into a high-quality output, which is more like the process of human reflection and more in line with the process of reinforcement learning. More specifically, instead of manually annotating, we generate drafts on crude models, but guide changes on more elaborate models. Our method has obtained very good results on real data, which has found new research directions for LLM research especially in RLHF field.
Submission Number: 14
Loading