Identity Bridge: Enabling Implicit Reasoning via Shared Latent Memory

ICLR 2026 Conference Submission18306 Authors

Keywords: Two hop reasoning, large language model, implicit regularization

Abstract: Despite remarkable advances, large language models often fail at compositional reasoning tasks, a phenomenon exemplified by the “curse of two-hop reasoning". This paper introduces the Identity Bridge, a simple yet powerful mechanism that resolves this compositionality gap by supervising the model on a zero-hop identity task. We demonstrate empirically that this addition enables models to successfully perform out-of-distribution two-hop reasoning, a task they otherwise completely fail. To explain this phenomenon, we provide a theoretical analysis using a simplified Emb-MLP model, proving that identity supervision reshapes the model's latent geometry. We show this alignment is induced by an implicit nuclear-norm regularization during optimization, which favors low-rank solutions that share structure across tasks. For complex tasks, we use small initialization or weight decay to enhance the regularization effect, which enhances the latent space alignment effect and slows down the generalization decay. Finally, we extend our investigation to large-scale models, observing that they still achieve two-hop reasoning through the latent memory, which provides crucial inspiration for enhancing their implicit reasoning abilities.

Primary Area: interpretability and explainable AI

Submission Number: 18306