Concept Attractors in LLMs and their Applications

Sotirios Panagiotis Chytas, Vikas Singh

Keywords: Large Language Models, Dynamic Systems, Attractors, Guardrails, Transpiler, Steering, Hallucinations, Synthetic Data

Abstract: Large language models (LLMs) often map semantically related prompts to similar internal representations at specific layers, even when their surface forms differ widely. We show that this behavior can be generalized and explained through Iterated Function Systems (IFS), where layers act as contractive mappings toward concept-specific Attractors. We leverage this insight and develop simple, training-free methods that operate directly on these attractors to solve a wide range of practical tasks, including **language translation**, **hallucination reduction**, **guardrailing**, and **synthetic data generation**. Despite their simplicity, these attractor-based interventions match or exceed specialized baselines, offering an efficient alternative to heavy fine-tuning, generalizable in scenarios where baselines underperform.

Primary Area: Applications (e.g., vision, language, speech and audio, Creative AI)

Submission Number: 13419