Identity Bridge: Enabling Implicit Reasoning via Shared Latent Memory

Pengxiao Lin, Zheng-An Chen, Zhi-Qin John Xu

Keywords: Two hop reasoning, large language model, implicit regularization

Abstract: Despite remarkable advances, large language models often fail at compositional reasoning tasks, a phenomenon exemplified by the ``curse of two-hop reasoning''. The inability of large models to perform implicit reasoning as expected remains an intriguing mystery. In this work, we unexpectedly discover that even a simple, single-layer Emb-MLP model can effectively learn to generalize out-of-distribution (OOD) multi-hop reasoning. Specifically, we demonstrate that transformers can successfully address multi-hop tasks through simple shortcuts rather than implicit reasoning with the help of Identity bridge. Our experiments and theoretical analysis established the learning mechanisms of the Emb-MLP model, which is the excellent approximation of GPT-2 on multi-hop task. We further compare the performance of the Emb-MLP model with GPT-2 across various complexities of two-hop reasoning and identify that smaller initializations or larger weight decay parameters enhance the model’s ability to harness implicit reasoning. Finally, we generalize our observations to real-world LLMs, presenting evidence that they implicitly acquire this necessary alignment during pretraining, which underpins their compositional abilities.

Primary Area: interpretability and explainable AI

Submission Number: 18306