Hint: harnessing the wisdom of crowds for handling multi-phase tasks

Yili Fang, Pengpeng Chen, Tao Han

Abstract: The resourcefulness of crowdsourcing can be used to handle a wide range of complex macro-tasks, such as travel planning, translation, and software development. Multi-phase tasks are a type of macro-task that consists of several subtasks distributed across multiple sequential phases. Due to the recent work’s disregard for the task’s sequential correlation, it is difficult for them to handle multi-stage tasks effectively. This work bridges this gap. We call this novel approach Hint, which incorporates task design, pre hoc worker coordination, and post hoc crowd work coordination. Starting with the task interface design, Hint makes workers aware of the relationship between phases in order to improve their processing abilities. Second, pre hoc coordination of workers is to organize the workers to do the tasks to lower the monetary costs required to meet a specific quality standard. Third, post hoc coordination of crowd work is through a decision tree-based coordination strategy. Extensive tests are carried out on real-world datasets to validate the desirable qualities of the suggested mechanism.