LLMs Know More Than Words: A Genre Study with Syntax, Metaphor & Phonetics

ACL ARR 2026 January Submission522 Authors

Keywords: Multilingual NLP, Literary Genre Classification, Stylistic Analysis, Pre-trained Language Models, Model Analysis, Cross-lingual Evaluation

Abstract:

Large language models (LLMs) demonstrate remarkable potential across diverse language‑related tasks, yet whether they capture deeper linguistic properties—such as syntactic structure, phonetic cues, and metrical patterns—from raw text remains unclear. To analysis whether LLMs can learn these features effectively and apply them to important nature language related tasks, we introduce a novel multilingual genre classification dataset derived from Project Gutenberg, a large-scale digital library offering free access to thousands of public domain literary works, comprising thousands of sentences per binary task (poetry vs. novel; drama vs. poetry; drama vs. novel) in six languages (English, French, German, Italian, Spanish, and Portuguese). We augment each with three explicit linguistic feature sets (syntactic tree structures, metaphor counts, and phonetic metrics) to evaluate their impact on classification performance. Experiments demonstrate
that although LLM classifiers can learn latent linguistic structures either from raw text or from explicitly provided features, different features contribute unevenly across tasks, which underscores the importance of incorporating more complex linguistic signals during model training.

Paper Type: Long

Research Area: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Research Area Keywords: rhetoric and framing, style analysis, multilingual evaluation, metaphor, morphological analysis

Contribution Types: Model analysis & interpretability, Data analysis

Languages Studied: English, French, German, Italian, Spanish, Portuguese

Submission Number: 522