TARA: Token-level Attribute Relation Adaptation for Multi-Attribute Controllable Text Generation

Yilin Cao, Jiahao Zhao, Ruike Zhang, Hanyi Zou, Wenji Mao

Abstract: Multi-attribute controllable text generation (CTG) aims to generate fluent text satisfying multiple attributes, which is an important and challenging task. The majority of previous research on multi-attribute CTG has ignored the interrelations of attributes that affect the performance of text generation. Recently, several work considers the attribute relations by explicitly defining them as inhibtory. We argue that for multi-attribute CTG, the attribute relations are not fixed, which can be not only inhibtory but promotive as well. In this paper, we tackle the multi-attribute CTG problem by explicitly identifying the above attribute relations for the first time and propose TARA, which employs token-level attribute relation adaptation and representation to generate text with the balanced multi-attribute control. Experimental results on the benchmark dataset demonstrate the effectiveness of our proposed method.