Q-Hitter: A Better Token Oracle for Efficient LLM Inference via Sparse-Quantized KV Cache

Download PDF
Open Webpage

Zhenyu Zhang, Shiwei Liu, Runjin Chen, Bhavya Kailkhura, Beidi Chen, Atlas Wang

Published: 2024, Last Modified: 27 Sept 2024MLSys 2024EveryoneRevisionsBibTeXCC BY-SA 4.0