TurboCache: Empowering Switch-Accelerated Key-Value Caches with Accurate and Fast Cache Updates

Xiang Chen, Longlong Zhu, Linying Zheng, Lingfei Cheng, Jianshan Zhang, Xu Yang, Dong Zhang, Xuan Liu, Xiaoming Lu, Xun Yi, Ibrahim Khalil, Albert Y. Zomaya, Haifeng Zhou, Chunming Wu

Abstract: Recent key-value (KV) caches are offloaded to programmable switches to offer high query processing performance. However, they suffer from both low accuracy in hot key detection and high latency in cache updates due to the strict limitations on switch registers. We propose TurboCache, a switch-accelerated KV cache with accurate hot key detection and fast cache updates. Our key idea is to leverage the switch recirculation capability to build a novel data structure that caches hot KV pairs. With this hardware-compatible cache data structure, TurboCache designs efficient data plane algorithms that accurately detects new hot keys and quickly updates its cache entirely within switch ASIC pipelines. We have implemented TurboCache on a ${64}\times {100}$ Gbps Tofino switch. Testbed results indicate that TurboCache improves the hot key detection accuracy and decreases the cache update latency of existing KV caches by several orders of magnitude.