Answer Aggregation for Crowdsourcing Microtasks using Approximate Global Optimal Searching

Lizhen Cui, Jing Chen, Wei He, Wei Guo

Abstract: In micro-task crowdsourcing, the answers collected from the crowd are ambiguous and the final answer aggregation is challenging due to the various capabilities and knowledge background of the voluntary participants on the Internet. In this paper, we extend the local optimal result of Expectation-Maximization(EM) approach and propose an approximate global optimal algorithm for answer aggregation of crowdsourcing microtasks with binary answers. Our algorithm is expected to improve the accuracy of real answer estimation through further likelihood maximization. We conduct extensive experiments on both simulated and real-world datasets, and the experimental results illustrate that the proposed approach can obtain better estimation results and has higher performance than regular EM-based algorithms.