Distributed retrieval for massive remote sensing image metadata on spark

Fengyang Wang, Xuezhi Wang, Wenjuan Cui, Xiao Xiao, Yuanchun Zhou, Jianhui Li

Abstract: The massive data is constantly and rapidly growing in remote sensing field. How to achieve efficient storage and rapid retrieval of massive remote sensing image metadata is a difficult problem. Big Data technologies provide convenient and fast tools for the storage, retrieval and analysis of remote sensing image metadata. According to the feature of remote sensing image metadata, we first give new concepts of fat grid and thin grid and propose a grid indexing method called Spark-Fat-Thin-Grid-Index (SFTGridIndex). In SFTGridIndex, the index file and partitions files are stored in HDFS. We then design an optimized retrieval method based on SFTGrid-Index. The method can avoid the intersection computing of a large number of polygons. This research can ensure efficient storage and fast query of massive remote sensing image metadata and has good scalability.