MapReduce-based Closed Frequent Itemset Mining with Efficient Redundancy Filtering

Su-Qi Wang, Yu-Bin Yang, Yang Gao, Guang-Peng Chen, Yao Zhang

Abstract: Mining closed frequent item set(CFI) plays a fundamental role in many real-world data mining applications. However, memory requirement and computational cost have become the bottleneck of CFI mining algorithms, particularly when confronting with large scale datasets, which herewith makes mining closed frequent item set from large scale datasets a significant and challenging issue. To address the above issue, a parallelized AFOPT-close algorithm is proposed and implemented in this paper based on the cloud computing framework MapReduce, which is widely used to cope with large scale data. Furthermore, an efficient parallelized method for checking if a frequent item set is globally closed is also proposed on the MapReduce platform to further improve the mining performance. Experimental results are then provided and analyzed to verify the efficiency and effectiveness of the proposed methods for mining closed frequent item set.