Evaluating Performance Trade-offs of Caching Strategies for AI-Powered Querying Systems

Hyunju Oh, Wei Zhang, Christopher D. Rickett, Sreenivas R. Sukumar, Suren Byna

Abstract: With the rapid growth of accumulated data from various scientific domains, traditional data management systems face challenges in supporting complicated queries, such as pattern search, on massive amounts of data. To serve sophisticated queries through capturing precise features from data, recent data management systems seek to use artificial intelligence (AI) within the querying process. However, the characteristic of AI inference workflow within the querying process, such as intensive computation and expensive requirements for computing resources, becomes a bottleneck of the AI-powered query systems.In this paper, we provide a generalization of AI inference workflow in the context of AI-powered data discovery and we introduce three different caching strategies corresponding to each stage in the AI inference workflow. We provide in-depth performance evaluation on the impact of these caching strategies through a series of strong scaling experiments. Our experimental results show that the AI-powered data querying performance can be significantly improved by applying different caching strategies.