Crowdsourcing and Crowd Work

Jeffrey P. Bigham, Chinmay Kulkarni, Walter S. Lasecki

Abstract: Crowdsourcing and human computation are useful in a number of real-world applications. Crowds generate large data sets useful for natural language processing and computer vision; they work together to formulate intelligent responses far beyond what we can automate; and they power intelligent interactive systems currently impossible with automated approaches alone. In this course, affendees will learn how to work with the crowd to enable research and practical applications. They will gain experience from the worker's perspective, receive an introduction to writing programs that work with existing sources of crowds, (e.g., Amazon Mechanical Turk), apply usability principles for designing crowd tasks that elicit high-quality responses, use statistical methods to improve the quality of the work received, build systems that interface with crowd labor in real time, and conduct experiments to improve understanding of the differences between different sources of crowd work. The course will provide hands-on activities on each of these topics, and provide pointers to material that can provide more in depth information.