AdpDM: Adaptive Data Model for Efficient Dynamic Management of Large-Scale High-Cardinality Time-Series Databases

Ziyue Xu, Sutong Huang, Di Fei, Liping Yi, Chenfei Zhou, Gang Wang, Xiaoguang Liu, Xinyu Liu, Wenqing Yu, Zijing Wei, Shaozhi Liu

Abstract: With advancements in Internet of Things (IoT) and performance monitoring, time-series database have garnered significant attention. Time-series consists of two components: series keys and data points. Prior efforts focus on reducing the storage cost of data points. However, as the scale of data grows rapidly, storing series keys and their indexes consumes at most 75% storage costs, still leading to unacceptably high query latency. Furthermore, the growing cardinality of time-series also slows down the construction of real-time indexes. Previous methods often rely on a fixed data model or user-selected hot data mechanism for indexing, resulting in a small number of accessed indexes and high query latency.