Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention

Download PDF
Open Webpage

Bin Gao, Zhuomin He, Puru Sharma, Qingxuan Kang, Djordje Jevdjic, Junbo Deng, Xingkun Yang, Zhou Yu, Pengfei Zuo

Published: 2024, Last Modified: 28 Jan 2026USENIX ATC 2024EveryoneRevisionsBibTeXCC BY-SA 4.0