Multi-Scale Manifold Alignment: A Unified Framework for Enhanced Explainability of Large Language Models

ACL ARR 2025 May Submission6171 Authors

Abstract: Recent advances in Large Language Models (LLMs) have achieved strong performance, yet their internal reasoning remains opaque, limiting interpretability and trust in critical applications. We propose a novel Multi-Scale Manifold Alignment framework that decomposes the latent space into global, intermediate, and local semantic manifolds—capturing themes, context, and word-level details. Our method introduces cross-scale mapping functions that jointly enforce geometric alignment (e.g., Procrustes analysis) and information preservation (via mutual information constraints like MINE or VIB). We further incorporate curvature regularization and hyperparameter tuning for stable optimization. Theoretical analysis shows that alignment error, measured by KL divergence, can be bounded under mild assumptions. This framework offers a unified explanation of how LLMs structure multi-scale semantics, advancing interpretability and enabling applications such as bias detection and robustness enhancement.

Paper Type: Long

Research Area: Semantics: Lexical and Sentence-Level

Research Area Keywords: Multi-Scale Manifold Alignment ，global, intermediate, and local semantic manifolds

Contribution Types: Model analysis & interpretability, Theory

Languages Studied: English

Submission Number: 6171