ShapeShifter: Workload-Aware Adaptive Evolving Index Structures Based on Learned Models

Hui Wang, Xin Wang, Jiake Ge, Lei Liang, Peng Yi

Track: Systems and infrastructure for Web, mobile, and WoT

Keywords: index structure, learned model, evolving strategy, data management, web search engine

Abstract:

In applications such as data management and Web search engines, indexes are key to enabling efficient data retrieval.
We find that unlike standard benchmarks with uniform data distribution, index operations in real-world tasks often exhibit strong skewness.
However, existing high-performance learned indexes, while proposed to enhance query and update efficiency, often fail to account for the characteristics of skewed workload access, leading to an imbalanced focus on optimizing a single performance metric at the expense of other critical aspects of overall index performance.
Furthermore, the complete use of learned models in index structures can lead to increased robustness issues, making them highly vulnerable to attacks and resulting in system unavailability.

To address these challenges, we propose ShapeShifter, an adaptive evolutionary structure based on traditional indexes, capable of dynamically adjusting node structures according to the workload.
ShapeShifter introduces a node evolution strategy, designed with workload-skew-aware policies, to adaptively adjust and optimize the most suitable partial index structure, leveraging a hybrid mechanism that combines traditional and learned structures for robust performance and an optimal time-space trade-off under skewed workloads and extreme data conditions.
The evaluation results show that ShapeShifter achieves the optimal trade-off between performance and space efficiency while maintaining robustness.

Submission Number: 1973