A Bi-Objective $\epsilon $-Constrained Framework for Quality-Cost Optimization in Language Model Ensembles

Aditya Singh, Aditi Singla, Kanishk Kukreja

Keywords: Large language models, Cost Optimization, LLM Ensembling

TL;DR: An ensembling framework that utilizes various open-sourced LLMs to achieve superior response quality at reduced costs through a bi-objective optimization approach and budget constraint simplification.

Abstract: We propose an ensembling framework that uses diverse open-sourced Large Language Models (LLMs) to achieve high response quality while maintaining cost efficiency. We formulate a bi-objective optimization problem to represent the quality-cost tradeoff and then introduce an additional budget constraint that reduces the problem to a straightforward 0/1 knapsack problem. We empirically demonstrate that our framework outperforms the existing ensembling approaches in response quality while significantly reducing costs.

Submission Number: 216