SimilarHITs: Revealing the Role of Task Similarity in Microtask Crowdsourcing
Abstract: Workers in microtask crowdsourcing systems typically consume different types of tasks. Task consumption is driven by the self-selection of workers in the most popular platforms such as Amazon Mechanical Turk and CrowdFlower. Workers typically complete tasks one after another in a chain. Prior works have revealed the impact of ordering tasks while considering aspects such as task complexity. However, little is understood about the benefits of considering task similarity in microtask chains. In this paper, we investigate the role of task similarity in microtask crowdsourcing and how it affects market dynamics. We identified different dimensions that affect the perception of task similarity among workers, and propose a supervised machine learning model to predict the overall task similarity of a task pair. Leveraging task similarity, we studied the effects of similarity on worker retention, satisfaction, boredom and fatigue. We reveal the impact of chaining tasks according to their similarity on worker accuracy and their task completion time. Our findings enrich the current understanding of crowd work and bear important implications on structuring workflow.
Loading