Classifiers are Better Experts for Controllable Text GenerationDownload PDF

Anonymous

04 Mar 2022 (modified: 05 May 2023)Submitted to NLP for ConvAIReaders: Everyone
Keywords: text generation, controllable text generation, language model
TL;DR: Guiding language model with a free-form classifier for controllable text generation
Abstract: This paper proposes a simple method for controllable text generation based on importance sampling, namely CAIF sampling. Using an arbitrary third-party text classifier, we adjust a small part of a language model's logits and guide text generation towards or away from classifier prediction. We show that the proposed method significantly outperforms recent PPLM, GeDi, and DExperts on PPL and sentiment accuracy based on the external classifier of generated texts. A the same time, it is also easier to implement and tune, and has significantly fewer restrictions and requirements.
0 Replies

Loading