Tabular Learning with Background Information: LLMs, Knowledge Graphs, or Both?

ICLR 2026 Conference Submission20375 Authors

Keywords: Tabular Learning, Large Language Models, Knowledge Graphs

TL;DR: Extensive experiments on real-life datasets show that knowledge-rich representations boost tabular learning, the most promising alley being to refine LLMs on Knowledge Graphs

Abstract: Tables have their own structure, calling for dedicated tabular learning methods with the right inductive bias. These methods outperform language models. Yet, many tables contain text that refers to real-world entities, and most tabular learning methods ignore the external knowledge that such strings could unlock. Which knowledge-rich representations should tabular learning leverage? While large language models (LLMs) encode implicit factual knowledge, knowledge graphs (KGs) share the relational structure of tables and come with the promise of better-controlled knowledge. Studying tables in the wild, we assemble 105 tabular learning datasets comprising text. We find that knowledge-rich representations, from LLMs or KGs, boost prediction, and combined with simple linear models they markedly outperform strong tabular baselines. Larger LLMs provide greater gains, and refining language models on a KG boosts models slightly. On datasets where all entities are linked to a KG, LLMs and KG models of similar size perform similarly, suggesting that the benefit of LLMs over KGs is to solve the entity linking problem. Our results highlight that external knowledge is a powerful but underused ingredient for advancing tabular learning, with the most promising direction lying in the combination of LLMs and KGs.

Primary Area: datasets and benchmarks

Submission Number: 20375