Bench-CoE: A Framework for Collaboration of Experts from Benchmark

ICLR 2026 Conference Submission16362 Authors

Keywords: Bench-CoE, Framework, Collaboration of Experts, Benchmark, LLMs

TL;DR: We propose the Bench-CoE framework that enables Collaboration of Experts (CoE) by effectively leveraging benchmark evaluations to achieve optimal performance across various tasks.

Abstract: Large Language Models (LLMs) are key technologies that drive intelligent systems to handle multiple tasks. To meet the demands of various tasks, an increasing number of LLMs-driven experts with diverse capabilities have been developed, spreading from language to visual understanding and generalization, accompanied by corresponding benchmarks to evaluate their performance. This paper proposes the Bench-CoE framework, which enables Collaboration of Experts (CoE) by effectively leveraging benchmark evaluations to achieve optimal performance across various tasks. Bench-CoE consists of a set of specialized expert models, a router for assigning tasks to corresponding experts, and a benchmark dataset for training the router. Based on this framework, we first formulate Query-Level Bench-CoE that is an abstraction of existing CoE methods exploiting the benchmark dataset. We further propose Subject-Level Bench-CoE, a new method that effectively addresses the potential issues of Query-Level Bench-CoE in poor generalization and labeling costs during training the router. Experiments show that the Query-Level Bench-CoE excels in in-distribution tasks, while the Subject-Level Bench-CoE demonstrates stronger out-of-distribution generalization. Our proposed Bench-CoE achieves efficient expert collaboration with minimal training label costs, improving adaptability in multi-task and cross-domain scenarios.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 16362