Stress Testing Generalization: How Minor Modifications Undermine Large Language Model Performance

ACL ARR 2025 February Submission7359 Authors

Abstract: This paper investigates the fragility of Large Language Models (LLMs) in generalizing to novel inputs, specifically focusing on minor perturbations in well-established benchmarks (e.g., slight changes in question format or distractor length). Despite high benchmark scores, LLMs exhibit significant accuracy drops and unexpected biases (e.g., preference for longer distractors) when faced with these minor but content-preserving modifications, highlighting their limited generalization capabilities. This analysis suggests that LLMs rely heavily on superficial cues rather than forming robust, abstract representations that generalize across formats, lexical variations, and irrelevant content shifts. This work aligns with the ACL 2025 theme track on the 'Generalization of NLP models,' proposing a “Generalization Stress Test” to assess performance shifts under controlled perturbations. The study calls for reevaluating benchmarks and developing more reliable evaluation methodologies to capture LLM generalization abilities better.

Paper Type: Short

Research Area: Special Theme (conference specific)

Research Area Keywords: Language Models

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 7359