Q-Hitter: A Better Token Oracle for Efficient LLM Inference via Sparse-Quantized KV Cache

Published: 2024, Last Modified: 27 Sept 2024MLSys 2024EveryoneRevisionsBibTeXCC BY-SA 4.0
Loading