Accelerating Iterative Retrieval-augmented Language Model Serving with Speculation

Open Webpage

Zhihao Zhang, Alan Zhu, Lijie Yang, Yihua Xu, Lanting Li, Phitchaya Mangpo Phothilimthana, Zhihao Jia

Published: 01 Jan 2024, Last Modified: 16 May 2025ICML 2024EveryoneRevisionsBibTeXCC BY-SA 4.0