SIMD-Based Multiple Sets Intersection with Dual-Scale Search Algorithm

Xingshen Song, Yuexiang Yang, Xiaoyong Li

Abstract: Conjunctive Boolean query is one fundamental operation for document retrieval in many information systems and databases. Various algorithms have been put up in terms of maximizing the query efficiency. In recent years, researchers began to exploit the parallel advantage of single-instruction-multiple-data (SIMD) instructions to accelerate the intersection procedure and achieved substantial gains over previous scalar algorithms. However, these works only focus on intersecting two sets at a time and ignore the scenario of multiple sets intersection. We present a flexible search algorithm which balances non-SIMD and SIMD comparisons in order to provide efficient and effective intersection.