Large-scale multimedia data mining using MapReduce framework

Hanli Wang, Yun Shen, Lei Wang, Kuangtian Zhufeng, Wei Wang, Cheng Cheng

Abstract: In this paper, the framework of MapReduce is explored for large-scale multimedia data mining. Firstly, a brief overview of MapReduce and Hadoop is presented to speed up large-scale multimedia data mining. Then, the high-level theory and low-level implementation for several key computer vision technologies involved in this work are introduced, such as 2D/3D interest point detection, clustering, bag of features, and so on. Experimental results on image classification, video event detection and near-duplicate video retrieval are carried out on a five-node Hadoop cluster to demonstrate the efficiency of the proposed MapReduce framework for large-scale multimedia data mining applications.