Epic-Level Text Generation with LLM through Auto-prompted Reinforcement Learning

Qianqian Qi, Lin Ni, Zhongsheng Wang, Libo Zhang, Jiamou Liu, Michael Witbrock

Abstract: In an era where the capabilities of large language models (LLM) like ChatGPT are transforming digital communication, the challenge of directing these tools to create extensive, coherent narratives on an epic-scale has emerged as a critical frontier. This study introduces a novel methodology that fuses the spontaneous story generation of LLMs with the precision of auto-prompted reinforcement learning for crafting epic-scale, coherent narratives. Our approach starts with generating a skeletal outline, followed by iterative expansion, and blending operations for maintaining structural coherence in long-form content. To train the reinforcement learning model efficiently, we introduce an environment simulator that leverages a database of historical LLM interactions, circumventing the limitations of direct LLM interactions. This method enhances the decision-making process of the RL agent, enabling more effective prompt selection and narrative flow in extended texts. We validate its effectiveness through experiments, demonstrating the model’s ability to generate structured, narrative-driven text, thereby setting a new pathway towards AI-driven, large-scale storytelling.