Optimizing the Effectiveness-Efficiency-Interpretability Trade-off in Text Classification via Hard Instance Paraphrasing

ACL ARR 2026 January Submission6992 Authors

Keywords: text classification, large language model, paraphrasing, rewrite, Effectiveness, Efficiency, Interpretability

Abstract: Large Language Models (LLMs) achieve strong text classification performance but incur high training and inference costs. Small Language Models (SLMs) are more efficient yet struggle with ambiguous or low-confidence ("hard'') instances. We propose RAP -- Rewrite-Assess-Predict for Hard Instances, a multi-stage framework that combines SLM efficiency with zero-shot LLM adaptability. RAP first detects hard instances using calibrated confidence, then applies prompt-optimized LLM paraphrasing to make class-relevant cues more explicit, with the resulting paraphrases replacing the original texts for subsequent classification. The paraphrased texts are reassessed, and predictions are chosen using calibrated confidence, ensuring performance does not degrade when rewriting is unhelpful. Experiments across multiple datasets show that paraphrases preserve semantic content, mitigate label shifts, and substantially improve SLM effectiveness on hard cases. Qualitative analysis further reveals systematic textual modifications that enhance class salience and interpretability. Overall, RAP provides a more accurate, reliable, interpretable, scalable, and cost-effective alternative to end-to-end LLM classification.

Paper Type: Long

Research Area: Machine Learning for NLP

Research Area Keywords: knowledge-augmented methods; generative models; data augmentation; few-shot learning;

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches to low-resource settings, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models, Data analysis

Languages Studied: english

Submission Number: 6992