Meta-Tuning LLMs to Elicit Lexical Knowledge of Language Style

Anonymous

TL;DR: Meta-tuning large language models (LLMs) with style lexicons can enhance their ability to recognize and adapt to new language styles in a zero-shot transfer setting.

Abstract: Language style is often used by writers to convey their intentions, identities, and mastery of languages. In this paper, we show that current large language models struggle to capture some of the language styles without fine-tuning. To address this challenge, we investigate whether LLMs can be meta-trained based on representative lexicons to recognize new language styles that they have not been fine-tuned on. Experiments on 13 established style classification tasks, as well as 63 novel tasks generated using LLMs, demonstrate that meta-training with style lexicons consistently improves zero-shot transfer across styles. Code and data to reproduce our experiments will be released upon publication.

Paper Type: long

Research Area: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Contribution Types: NLP engineering experiment, Approaches to low-resource settings

Languages Studied: English