Dynamic Manifold Evolution Theory: Modeling and Stability Analysis of Latent Representations in Large Language Models

ACL ARR 2025 May Submission5340 Authors

Abstract: We introduce Dynamic Manifold Evolution Theory (DMET), a unified framework that models large-language‐model generation as a controlled dynamical system evolving on a low-dimensional semantic manifold. By casting latent‐state updates as discrete‐time Euler approximations of continuous dynamics, we map intrinsic energy‐driven flows and context-dependent forces onto Transformer components (residual connections, attention, feed-forward networks). Leveraging Lyapunov stability theory We define three empirical metrics (state continuity, clustering quality, topological persistence) that quantitatively link latent‐trajectory properties to text fluency, grammaticality, and semantic coherence. Extensive experiments across decoding parameters validate DMET’s predictions and yield principled guidelines for balancing creativity and consistency in text generation.

Paper Type: Long

Research Area: Interpretability and Analysis of Models for NLP

Research Area Keywords: Dynamic Manifold Evolution Theory (DMET),dynamical system

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 5340