TeleCrowd: A Crowdsourcing Approach to Create Informal to Formal Text Corpora

Vahid Masoumi, Mostafa Salehi, Hadi Veisi, Golnoush Haddadian, Vahid Ranjbar, Mahsa Sahebdel

Abstract:

Crowdsourcing has been widely used recently as an alternative to traditional
annotations that is costly and usually done by experts. However, crowdsourcing tasks
are not interesting by themselves, therefore, combining tasks with game will increase
both participant’s motivation and engagement. In this paper, we have proposed a
gamified crowdsourcing platform called TeleCrowd based on Telegram Messenger to
use its social power as a base platform and facilitator for accomplishing
crowdsourcing projects. Furthermore, to evaluate the performance of the proposed
platform, we ran an experimental crowdsourcing project consisting of 500 informal
Persian sentences in which participants were supposed to provide candidates that were
the formal equivalent of sentences or qualify other candidates by upvoting or
downvoting them. In this study, 2700 candidates and 21000 votes were submitted by
the participants and a parallel dataset using candidates with the highest points, sum of
their upvotes and downvotes, as the best candidates was built. As the evaluation,
BLEU score of 0.54 was achieved on the collected dataset which shows that our
proposed platform can be used to create large corpora. Also, this platform is highly
efficient in terms of time period and cost price in comparison with other related works,
because the whole duration of the project was 28 days at a cost of 40 dollars.