Performance Aware LLM Load Balancer for Mixed Workloads

Open Webpage

Kunal Jain, Anjaly Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Rühle, Anoop Kulkarni, Steve Kofsky, Saravan Rajmohan

Published: 01 Jan 2025, Last Modified: 15 May 2025EuroMLSys 2025EveryoneRevisionsBibTeXCC BY-SA 4.0