Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasing

ACL ARR 2024 December Submission1594 Authors

Abstract:

Dynamical systems theory provides a framework for analyzing iterative processes and evolution over time.
Within such systems, repetitive transformations can lead to stable configurations, known as attractors, including fixed points and limit cycles. Applying this perspective to large language models (LLMs), which iteratively map input text to output text, provides a principled approach to characterizing long-term behaviors. Successive paraphrasing serves as a compelling testbed for exploring such dynamics, as paraphrases re-express the same underlying meaning with linguistic variation. Although LLMs are expected to explore a diverse set of paraphrases in the text space, our study reveals that successive paraphrasing converges to stable periodic states, such as 2-period attractor cycles, limiting linguistic diversity. This phenomenon is attributed to the self-reinforcing nature of LLMs, as they iteratively favour and amplify certain textual forms over others. This pattern persists with increasing generation randomness or alternating prompts and LLMs. These findings underscore inherent constraints in LLM generative capability, while offering a novel dynamical systems perspective for studying their expressive potential.

Paper Type: Long

Research Area: Interpretability and Analysis of Models for NLP

Research Area Keywords: paraphrase generation, self-reinforcement, large language models

Languages Studied: Englihs, Chinese

Submission Number: 1594