LayerCake: Token-Aware Contrastive Decoding within Large Language Model Layers

ICLR 2026 Conference Submission13304 Authors

Keywords: Contrastive decoding, Large Language Models

Abstract: Large language models (LLMs) excel at natural language understanding and generation but remain vulnerable to factual errors, limiting their reliability in knowledge-intensive tasks. While decoding-time strategies provide a promising efficient solution without training, existing methods typically treat token-level and layer-level signals in isolation, overlooking the joint dynamics between them. In this work, we introduce a token-aware, layer-localized contrastive decoding method that aligns specific token types with their most influential transformer layers to improve factual generation. Through empirical attention analysis, we identify two key patterns: punctuation tokens receive dominant attention in early layers, while conceptual tokens govern semantic reasoning in intermediate layers. By selectively suppressing attention to these token types at their respective depths, we induce controlled factual degradation and derive contrastive signals to guide decoding. Our method requires no additional training or model modification, consistently improving factuality across multiple LLMs and various benchmarks.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 13304