TreeTop: Topology-Aware Fine-Tuning for LLM Conversation Tree Understanding

Jashn Arora, Rahul Madhavan, Karthikeyan Shanmugam, John Palowitch, Manish Jain

Keywords: Conversation Trees, Social media, Large language models

TL;DR: We study and develop the ability of large language models (LLMs) to understand and reason over structural relationships in conversations, especially on social media platforms.

Abstract: While Large Language Models (LLMs) have dominated a wide diversity of natural language tasks, improving their capabilities on \emph{structured} inputs such as graphs remains an open challenge. We introduce $\texttt{TreeTop}$, a pre-training framework for LLMs that significantly improves their ability to understand and reason over structural relationships in multi-party, threaded discussions, such as those found on social media platforms. $\texttt{TreeTop}$ is a novel set of 17 QA-style tasks specifically designed to allow LLMs to selectively focus on both the structure of and content in discussion graphs. We find that LLMs fine-tuned with $\texttt{TreeTop}$ outperform their counterparts in every setting: zero-shot/few-shot performance on unseen pretraining tasks as well as downstream social media inference tasks (e.g.rumor detection), as well as fine-tuned performance on the downstream tasks, including their challenging "early-detection" variants. In particular, $\texttt{Gemini Pro}$ fine-tuned with $\texttt{TreeTop}$ and further fine-tuned on downstream tasks surpasses both vanilla $\texttt{Gemini Pro}$ and state-of-the-art GNN baselines. Our framework paves the way for LLMs with enhanced capabilities on heavily-structured inputs.

Primary Area: foundation or frontier models, including LLMs

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6468