Beyond Monoliths: Expert Orchestration for More Capable, Democratic, and Safe Large Language Models

Philip Quirke, Narmeen Fatimah Oozeer, Amir Abdullah, Jason Hoelscher-Obermaier, Clement Neo, Jeff M. Phillips, Joshua Greaves, Chaithanya Bandi, Fazl Barez, Shriyash Kaustubh Upadhyay

Keywords: LLM, judge, router, ensemble, meta-model, transparency, trust, experts

TL;DR: Replace the current reliance on monolithic general-purpose LLMs with a "judges and routers" architecture that coordinates specialist and generalist models to deliver more accurate, cost-effective, and user-aligned AI responses.

Abstract:

This position paper argues that the prevailing trajectory toward ever larger, more expensive generalist foundation models controlled by a handful of big companies limits innovation and constrains progress.
We challenge this approach by advocating for an “Expert Orchestration" architecture as a superior alternative that democratizes LLM advancement.
Our proposed architecture intelligently selects from thousands of existing models based on query requirements and decomposition, focusing on identifying what models do well rather than how they work internally.
Independent “judge" models assess various models' capabilities across dimensions that matter to users, while “router" systems direct queries to the most appropriate specialists within an approved set.
This approach delivers superior performance by leveraging targeted expertize rather than forcing costly generalist models to address all user requirements.
The expert orchestration paradigm represents a significant advancement in LLM capability by enhancing transparency, control, alignment, and safety through model selection while fostering a more democratic ecosystem.

Submission Number: 63