An Efficient Segment Grouping Approach for Active Disk-Based Storage Systems

Quanqing Xu, Weiya Xi, Khai Leong Yang, Chao Jin

Abstract: Active storage is an excellent big data storage and analysis platform by exploring storage nodes' computing resources since it especially fits read-intensive operations. Active disks have a similar idea with active storage except utilizing the computing resources embedded in disk drives. Active disk-based storage systems have to evenly distribute data and workload among many storage devices so that we could efficiently use available resources to maximize system performance. We propose an efficient Segment Grouping (SG) approach in this paper. SG is a pseudo-random data distribution approach that is designed for active disk-based storage architecture. It is excellent at load balancing because it utilizes hash functions. Any party can independently compute the location of any data segment. SG efficiently maps segment objects to storage devices using the hash functions, and it only needs a compact and hierarchical description of storage devices in the active disk-based storage system. It only needs knowledge of the segment placement policy, and all the required metadata is little and mostly static. Our experimental results demonstrate that it is efficient and scalable by conducting performance evaluation.

External IDs:dblp:conf/icpads/XuXYJ18