Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer

Zhiqiang Hu, Nancy F. Chen, Roy Ka-Wei Lee

Submission Type: Regular Long Paper

Submission Track: Efficient Methods for NLP

Submission Track 2: Natural Language Generation

Keywords: Text style transfer, Parameter-efficient, Adapter

Abstract: Adapting a large language model for multiple-attribute text style transfer via fine-tuning can be challenging due to the substantial amount of computational resources and labeled data required for the specific downstream task. In this paper, we address this challenge by introducing \textsf{Adapter-TST}, a framework that freezes the pre-trained model's original parameters and enables the development of a multiple-attribute text style transfer model. Using BART as the backbone model, \textsf{Adapter-TST} utilizes different neural adapters to model different types of attribute information, similar to a plug-in connected to BART. Our method allows control over multiple attributes (e.g. sentiment, tense, active or passive voice) and configures the adapters' architecture to generate multiple outputs in respect to attributes or compositional editing on the same sentence. We evaluate the proposed model on both traditional sentiment transfer and multiple-attribute transfer tasks. The experiment results demonstrate that \textsf{Adapter-TST} outperforms all the state-of-the-art baselines with significantly less computational resources. We have also empirically shown that each adapter is able to characterize specific stylistic attributes effectively and can be configured to perform compositional editing.

Submission Number: 1404