Incorporating Feature Labeling into Crowdsourcing for More Accurate Aggregation Labels

Yili Fang, Zhaoqi Pei, Xinyi Ding, Wentao Xu, Tao Han

Abstract: Crowdsourcing is a popular way of collecting crowd wisdom and has been deployed in various senarios. Effective answer collection and answer aggregation are two important crowdsourcing topics as workers may give incorrect responses. For difficult tasks, workers tend to implicitly use task related information during answer collection, and those information could play an important role in aggregating high-quality results. For example, the identification of the size and hair style of one dog in a picture is a simple and necessary prerequisite step for dog breed labeling. However, most existing methods ignore those task related information and fail to achieve high quality data.