AVPS: Automatic Vertical Partitioning for Dynamic Workload

Pengju Liu, Pan Cai, Cuiping Li, Hong Chen

Abstract: Vertical partitioning aims at optimizing data placement to accelerate queries by analyzing query access patterns. However, in practice, queries evolve over time, making it challenging to collect representative queries to construct robust partitions all at once. Existing methods rely on specific feature thresholds to decide when to repartition for adapting changing queries, but this approach does not directly quantify the potential benefits of repartitioning. Moreover, they generate new partitions based solely on recent historical queries to update underperforming old partitions. To address these challenges, we introduce an automatic vertical partitioning system (AVPS) that timely adjusts inefficient partitions. We first propose using a reinforcement learning algorithm, PPO, to identify the optimal re-partitioning timing. Specifically, we dynamically maintain an access matrix for the workload environment to represent the repartitioning urgency and use a trust-region policy to more accurately estimate the reward of re-partitioning. Next, we design a workload selection algorithm to identify valuable queries, which are then used by a partitioner to update old partitions. Experiments conducted on PostgreSQL show that our method outperforms existing solutions.