LLMs Read Nodes but No Graphs? Revisiting the Graph Modality Understanding in LLMs

ACL ARR 2025 May Submission6269 Authors

Abstract: Graphs naturally integrate semantic contents and interactional information into a unified representation, offering a clear and intuitive format for many objects such as molecules or citation/social networks. In parallel, large language models (LLMs) have demonstrated exceptional performance in understanding natural language and incorporating cross-modal information, leading to growing interest in their ability to process and reason over graphs. Recent studies have introduced LLMs to process graph-structured data by either designing parameter-free graph templates or applying graph neural networks (GNNs) to encode structural information. In this work, we were originally motivated to explore how different strategies for encoding structural information of graphs impact the capabilities of LLMs in the context of text-attributed graphs. Surprisingly, we observe in our systematic experiments that (i) LLMs using **only node textual descriptions already establish** a strong performance baseline for many tasks, and (ii) the majority of strategies for **incorporating structural information yields only marginal** or even negative performance gains. The finding calls into question the necessity of incorporating explicit graph structures for in the LLM era and suggests a rethinking of graph learning approaches when powerful language models are involved.

Paper Type: Short

Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond

Research Area Keywords: Multimodality, Graph Learning

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 6269