End-to-end Story Plot Generator

Hanlin Zhu, Andrew Cohen, Danqing Wang, Kevin Yang, Xiaomeng Yang, Jiantao Jiao, Yuandong Tian

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: automatic story generation, end-to-end generator, reader-specific reward model, rlhf

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We study end-to-end story plot generation, which is much faster than previous methods and easy to further fine-tune with human feedback.

Abstract: Story plots, while short, carry most of the essential information of a full story that may contain tens of thousands of words. We study the problem of automatic generation of story plots, which includes story premise, character descriptions, plot outlines, etc. To generate a single engaging plot, existing plot generators (e.g., DOC (Yang et al., 2022a)) require hundreds to thousands of calls to LLMs (e.g., OpenAI API) in the planning stage of the story plot, which is costly and takes at least several minutes. Moreover, the hard-wired nature of the method makes the pipeline non-differentiable, blocking fast specialization and personalization of the plot generator. In this paper, we overcome these issues with an end-to-end story plot generator, which is (1) faster and cheaper to generate and (2) end-to-end fine-tunable with human feedback. Compared to DOC, our work replaces expensive OpenAI API calls with Llama2 models via careful prompt designs, which leads to the cheap generation of high-quality training datasets. We then perform supervised fine-tuning (SFT) using approximately 13000 story plots to obtain an end-to-end model. The end-to-end model can generate story plots of comparable quality to the previous DOC method and is $>10\times$ faster (1k tokens in only 30 seconds on average). Furthermore, fine-tuned with RLHF on several different reward models for different aspects of story quality, our model achieves 60.0\% winning rate against the model after SFT in the aspect of suspense and surprise.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8614