DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving

Download PDF
Open Webpage

Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, Hao Zhang

Published: 01 Jan 2024, Last Modified: 01 Oct 2024OSDI 2024EveryoneRevisionsBibTeXCC BY-SA 4.0