IntuitiveGraphLLM: Intuitive Graph-based Text Representation with Large Language Model

Md Mostafizer Rahman, Yutaka Watanobe, Syed Rameez Naqvi, Lu Peng

Keywords: IntuitiveGraphLLM, IntuitiveGraph, Structure-aware, Graph, Graph Representation, Feature Fusion

TL;DR: We introduce Intuitive Graphs (IGs)—graphs that explicitly encode both (i) structural context and (ii) conceptual relevance—and IntuitiveGraphLLM, a framework that builds, encodes, and fuses IGs with pretrained LLMs.

Abstract: Graphical representations of text can sharpen the inductive biases of large language models (LLMs), yet most graph-based approaches rely on co-occurrence, order, or position alone and therefore over-connect unrelated tokens while missing conceptually salient links. We introduce Intuitive Graphs (IGs)—graphs that explicitly encode both (i) structural context (local order/proximity/position) and (ii) conceptual relevance (semantic affinity in embedding space)—and IntuitiveGraphLLM, a framework that builds, encodes, and fuses IGs with pretrained LLMs. Given a passage, we first construct IGs by pruning structure-induced edges with a semantic gate based on cosine similarity between token (or span) embeddings, yielding sparse, human-plausible graphs. We then obtain initial node features from contextual embeddings and apply Graph Attention Networks (GATs) to emphasize informative nodes/edges to produce graph-level features. Finally, we perform hybrid fusion by integrating graph-level embeddings with LLM-based contextual representations, enabling the model to leverage complementary structural and conceptual signals. We evaluate our approach on five benchmark datasets spanning short and long documents and class-imbalance settings. Across benchmarks, IntuitiveGraphLLM consistently improves over strong text-only and graph-only baselines; gains persist under varied IG constructions, node embeddings, GAT depths/heads, and LLM backbones, with ablations confirming that IG is the key driver of performance and reduced edge noise. IntuitiveGraphLLM provides a principled, interpretable way to make text graphs both contextual and conceptually grounded, translating into more faithful reasoning and stronger downstream accuracy.

Primary Area: learning on graphs and other geometries & topologies

Submission Number: 4550