Learning to Link: Incorporating Multi-hop QA Examples Improves Dispersed Knowledge Injection

ICLR 2026 Conference Submission21525 Authors

Keywords: language models, knowledge injection, multi-hop reasoning, question answering, cross-document linking

TL;DR: We demonstrate that language models trained with multi-hop QA examples grounded in documents learn to connect dispersed facts and generalize cross-document linking across domains using controlled fictional datasets.

Abstract: Language models have proven effective as knowledge bases for answering both single- and multi-hop questions at web scale. However, a persistent challenge is whether and how these models connect facts dispersed across documents --- a core requirement for multi-hop reasoning from parametric knowledge. In this paper, we present an empirical study of the learning dynamics underlying such linking in controlled settings. We compare different training regimes on varied synthetic datasets, showing that standard training on isolated documents leads to limited effectiveness in two-hop knowledge extraction. Our results indicate that interleaving exposure to documents and two-hop question answering (QA) examples --- whose answers require composing relations across documents --- enables models to generalize cross-document linking across domains, entities, and relations. A key finding is that QA examples alone are insufficient: pairing questions with their grounding documents during training is essential, indicating that models are not simply memorizing the QA format. Finally, we show that making these connections within a single forward pass remains challenging; therefore, chain-of-thought answering is crucial for assessing the injection of knowledge dispersed across documents.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 21525