GraphRouter: A Graph-based Router for LLM Selections
Keywords: LLM selection, Graph-based router, Contextual interactions, New LLM settings
TL;DR: We propose GraphRouter, a novel graph-based router that improves LLM selection for diverse tasks by leveraging an inductive graph framework to optimize performance-cost trade-offs and generalize across various models and scenarios.
Abstract: The rapidly growing number and variety of Large Language Models (LLMs)
present significant challenges in efficiently selecting the appropriate LLM for
a given query, especially considering the trade-offs between performance and
computational cost. Current LLM selection methods often struggle to generalize
across new LLMs and different tasks because of their limited ability to leverage
contextual interactions among tasks, queries, and LLMs, as well as their depen-
dence on a transductive learning framework. To address these shortcomings, we
introduce a novel inductive graph framework, named as GraphRouter, which
fully utilizes the contextual information among tasks, queries, and LLMs to en-
hance the LLM selection process. GraphRouter constructs a heterogeneous
graph comprising task, query, and LLM nodes, with interactions represented as
edges, which efficiently captures the contextual information between the query’s
requirements and the LLM’s capabilities. Through an innovative edge prediction
mechanism, GraphRouter is able to predict attributes (the effect and cost of
LLM response) of potential edges, allowing for optimized recommendations that
adapt to both existing and newly introduced LLMs without requiring retraining.
Comprehensive experiments across three distinct effect-cost weight scenarios have
shown that GraphRouter substantially surpasses existing routers, delivering a
minimum performance improvement of 12.3%. In addition, it achieves enhanced
generalization across new LLMs settings and supports diverse tasks with at least a
9.5% boost in effect and a significant reduction in computational demands. This
work endeavors to apply a graph-based approach for the contextual and adaptive
selection of LLMs, offering insights for real-world applications.
Primary Area: learning on graphs and other geometries & topologies
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 10977
Loading