DéjàVu: KV-cache Streaming for Fast, Fault-tolerant Generative LLM Serving

Open Webpage

Foteini Strati, Sara McAllister, Amar Phanishayee, Jakub Tarnawski, Ana Klimovic

Published: 01 Jan 2024, Last Modified: 11 May 2025ICML 2024EveryoneRevisionsBibTeXCC BY-SA 4.0