Evaluating the crowd quality for subjective questions based on a Spark computing environment

Xinxin Wang, Depeng Dang, Zixian Guo

Abstract: Highlights • An algorithm for the quality estimation based on subjective questions is proposed. • Specifically, confidence intervals are used to indicate personnel quality. • To improve its scalability and efficiency, algorithm is implemented on Spark. • Extensive experiments are conducted to demonstrate the effectivity. Abstract Crowdsourcing is a popular method of data collection or problem solving, which has been widely used in business, scientific research and other fields. Personnel quality control is a key issue in the field of crowdsourcing, and the method of estimating personnel quality is receiving more and more attention. This paper proposes a new algorithm for the quality estimation of crowdsourcing personnel, which is based on subjective questions innovatively. Specifically, confidence intervals are used to indicate personnel quality. To improve its scalability and efficiency in big data environment, algorithm is implemented on Spark, a widely-adopted distributed computing platform. Finally, extensive experiments are conducted on real-world data sets, and results demonstrate that algorithm can effectively and quickly estimate the quality of crowdsourcing personnel in the big data environment.