Multi-Agent Join

NeurIPS 2023 Workshop MLSys Submission17 Authors

Published: 28 Oct 2023, Last Modified: 12 Dec 2023MlSys Workshop NeurIPS 2023 PosterEveryoneRevisionsBibTeX
Keywords: Data Systems, Join Processing, Online Learning, Many-Armed Bandit, Multi-Agent Learning
Abstract: Real-time performance is crucial for interactive and exploratory data analysis, where users require quick access to subsets or progressive presentations of query results. Delivering real-time results over large data for common relational binary operators like join is challenging, as join algorithms often spend considerable time scanning and attempting to join parts of relations that may not produce any results. Existing solutions often involve repetitive preprocessing, which is costly and may not be feasible for interactive workloads or evolving datasets. Additionally, these solutions may support only restricted types of joins. This paper presents a novel approach for achieving efficient progressive join processing. The scan operator of the join learns online during query execution, identifying portions of its underlying relation that satisfy the join condition. Additionally, an algorithm is introduced where both scan operators collaboratively learn to optimize join execution.
Submission Number: 17