Concept Attractors in LLMs and their Applications

ICLR 2026 Conference Submission19458 Authors

Keywords: LLM, attractors, IFS, forgetting, synthetic data generation, hallucinations, translations

Abstract: Large language models (LLMs) often map semantically related prompts to similar internal representations at specific layers, even when their surface forms differ widely. We show that this behavior can be generalized and explained through Iterated Function Systems (IFS), where layers act as contractive mappings toward concept-specific Attractors. We leverage this insight and develop simple, training-free methods that operate directly on these attractors to solve a wide range of practical tasks, including **language translation, hallucination reduction, guardrailing**, and **synthetic data generation**. Despite their simplicity, these attractor-based interventions match or exceed specialized baselines, offering an efficient alternative to heavy fine-tuning, generalizable in scenarios where baselines underperform.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 19458