Large Language Models Enhance Graph Learning Without Graph Serialization

Ziv Mor, Roy Mahlab, Chaim Baskin, Moshe Eliasof

Keywords: Graph Neural Networks, Large Language Models, Message passing architectures

TL;DR: We suggest methodology and a method for enriching graphs with LLM-embedded prompt describing the graphs. No serialization needed.

Abstract:

We introduce \ourmethod, a framework that augments graph neural networks (GNNs) with global graph descriptions encoded by large language models (LLMs). While prior work on text-attributed graphs (TAGs) integrates text features tied to nodes or edges, our approach leverages external prompt datasets that describe graph semantics independently of the downstream task. These prompts are embedded by a frozen LLM and injected into the message-passing process via cross-attention, allowing GNNs to incorporate global contextual information during local computations. We provide a theoretical analysis showing that \ourmethod provably increases the expressive capacity of GNNs while preserving permutation equivariance.

Extensive experiments across diverse graph benchmarks demonstrate consistent improvements over strong GNN baselines, with performance gains of up to 10\%. Ablation studies further confirm that these improvements stem from the semantic content of the prompts rather than from random noise or model size. Together, our findings highlight LLM-embedded prompts as a principled and effective new modality for enhancing graph learning.

Primary Area: learning on graphs and other geometries & topologies

Submission Number: 3618