Text Style Transfer with Parameter-efficient LLM Finetuning and Round-trip Translation

ACL ARR 2025 February Submission5198 Authors

Abstract: This paper proposes a novel method for Text Style Transfer (TST) based on parameter-efficient fine-tuning of Large Language Models (LLMs). Addressing the scarcity of parallel corpora that map between styles, the study employs round-trip translation to synthesize such parallel datasets from monolingual corpora. This approach creates "neutralized" text devoid of stylistic attributes, essentially creating a shared input style at training-time and inference-time. Experimental results demonstrate consistent superiority of this method over zero-shot prompting and fewshot ICL techniques measured by BLEU scores and style accuracy scores across 6 investigated domains. Furthermore, the integration of retrieval-augmented generation (RAG) for terminology and name knowledge enhances robustness and stylistic consistency.

Paper Type: Long

Research Area: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Research Area Keywords: domain adaptation, style analysis, style generation, applications, data-efficient training, data augmentation, NLP in resource-constrained settings

Contribution Types: Approaches to low-resource settings, Data resources

Languages Studied: English, German, Chinese

Submission Number: 5198